diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 3657250b..2a6dbef7 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -2586,6 +2586,40 @@ semantics while widening the import surface the by-name refusal keeps closed; an interpreter module was declined because the pinned address would name the interpreter while the tool's logic rode in its input, outside every pin. +### A plan configures its model nodes; what reaches the provider rides the journal (#391–#394) + +**Decision (2026-10-20):** a Workflow's `llm` field configures each abstract +node: system text, instruction (literal or a pointer into the input), state +selection, a tool subset, structured output, temperature or none, the output +ceiling, `parallel_tool_calls`, and whether the attempt's tool calls reach the +nodes below. It also names tools a host declares in the run input +(`tools_from`), each dispatched through a Definition the plan pins. Three +rules carry the design. + +1. **Frozen at start, like `reads`.** The block, and the `tools_from` list + resolved from the input, are validated and frozen onto the node's pin. + Resume and `verify` run the frozen block, and neither the plan nor a later + input can change it. +2. **Every request field rides the journal.** The scheduler, which is pure, + writes the first message and each turn's `input.llm` (with + `response_format` in wire shape). The driver builds the request from that + journaled input alone, which is the rule the fold's no-tools marker and the + narrowed offer already follow. A node with no block journals what it always + did. +3. **The input names data, never code.** A dynamic tool may dispatch only + through a Definition the plan pinned, by a binding or by `llm.dispatch`. It + runs under that Definition's host pin and broker grants, with `locked` keys + the model cannot override, and journals under its own name with the + Definition's address. `locked_params` on a Definition, stored but never + applied before 1.13, follows the same merge. + +**Rejected:** per-run host flags for these knobs (the bindings had none, and a +plan-level setting is what a reviewer reads); offering `mcp.call` itself to the +model (one opaque tool, with the server and tool chosen by the model); and +letting `tools_from` name a Definition by hash that the plan does not pin, +which would let host data choose code. `docs/run.md` "Configuring an LLM node" +is the reference. + ### A host pause is a transient hold, never scheduler state (#344) **Decision (2026-09-23, #344):** a run gains a third stop beside cancel diff --git a/CHANGELOG.md b/CHANGELOG.md index ddaf99ac..1f2e3a7c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,118 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] +## [1.13.0] — 2026-10-20 + +### Added + +- **A plan configures each LLM node (`llm`)** (#391). A Workflow's new `llm` + field sets, per abstract node: `system` (replaces the built-in prompt); + `instruction` or `instruction_from`, a JSON pointer into the node's input + that may be nested, such as `/prompt/text`; `state` (`"all"`, `"none"` or a + list of keys); `tools` (a subset of the pinned host tools); `response` + (`{"json_schema": …}` or `{"json": true}`); `temperature` (a number, or + `"omit"` to send none, for the GPT-5 family); `max_tokens` (the node's + ceiling and reservation); `parallel_tool_calls`; and `tool_results_into`, + which hands every tool call of the attempt, built from the journal, to the + nodes below. The block is validated at start (`RUN-E019` for a shape error, + `RUN-E004` for an unpinned tool name) and frozen into the manifest, so + `run inspect` shows it and resume and `verify` use it. Each `mg:llm` intent + journals the request fields as sent under `input.llm`. The runtime checks a + structured answer whatever the provider did, with one corrective re-prompt + before the node fails `schema_validation_failed`. A node with no block + journals byte-identically, so the scheduler epoch did not move. +- **Tools declared in the run input (`tools_from`)** (#392). An abstract + node's `llm.tools_from` points at a list in the run input: + `{name, description, input_schema, dispatch: {definition, locked, + args_into}}`. The list is pinned at start and offered under its own names + and schemas. A call is validated against the entry's schema, wrapped under + `args_into`, locked, validated against the dispatch Definition's + `input_schema`, executed as that Definition (same pin, sandbox and broker + grants) and journaled under the entry's name. `llm.dispatch: [name | hash]` + pins a Definition that no node binds, such as `mcp.call`. It resolves as an + unbound node's Definition does and is never offered to the model by itself. + A dispatch target that is not pinned, a duplicate name, or a name that + collides with a pinned tool is refused at start, naming the entry. +- **Per-node tool-offer narrowing** (#396). `llm..narrow: false` keeps + a node's whole offer when a calibrated decision backend is pinned, and + `narrow: {"top_k": n}` sets the K for that node. Both are frozen in the + manifest, and `true` (the default) keeps the run-wide top-8 rule. The + narrowing question now describes a `tools_from` tool by the first line of + its own `description`, as it describes a pinned Definition by its + `tool_description`. Before this change it listed a dynamic tool by name + only. +- **`llm_profile` on the bindings** (#394). `runStart`/`runResume` (Node) and + `run_start`/`run_resume` (Python) take a trailing JSON string, + `{"temperature": "omit" | n, "token_field": …, "extra_body": {…}}`, so an + embedding host can run a model that refuses a temperature. + `ToolCallRequest` gains `response_format`, `parallel_tool_calls` and + `omit_temperature`, and now derives `Default`. `RequestProfile::from_json` + and `resolve_toolcall_with_profile` are new. + +### Fixed + +- **`locked_params` is applied** (#393). A Definition's locked parameters + were stored, validated and displayed, and never used: a model-issued call + was dispatched with whatever the model sent. Locks are now merged over the + model's arguments (the lock wins), stripped from the offered schema + (`properties` and `required`), checked by a strict Definition on the merged + input, journaled merged, and frozen with the pin. They are applied by run + dispatch (`areev-run`), the one place a model's tool call is executed. They + are not applied by `areev-loop`, `areev-server` or `areev-mcp`, which + dispatch no model-issued calls, nor to a bound host node. The issue's + acceptance read "merged at invoke", and invoke is run dispatch. A client or + decision Definition carrying `locked_params` still starts: the locks are + ignored, as in 1.12, with a `RUN-W001` warning in the manifest. `shadow` + rehearses a run with the locks that run froze, so a 1.12 run still scores + `same` on an unchanged plan. +- **Hardening from a review of the 1.13 branch:** + - Without `args_into`, a dynamic tool's schema is closed: a key the entry + does not name, or a key the host locked, is an argument violation. With + `args_into`, the entry's own schema decides, and only + `additionalProperties: false` closes it. + - The model's arguments are checked against the schema it was shown, so a + required key the host locks is never demanded. + - A dotted lock key, a lock on the `args_into` key, a dispatch pin that is + another version of a bound Definition's name, and a `json_schema` that + OpenAI's strict mode would refuse are all refused at start (`RUN-E019`). + - `{"json": true}` adds "Answer with a JSON object." to a node system text + that never says JSON. + - A fenced (```` ```json ````) final answer is accepted. + - The `tools_from` list is left out of the whole-state prompt. + - A node's `system` text, and a dynamic tool's description and schema + prose, are pseudonymized under an `egress` policy. + - Transports report the temperature they actually send, and a result + records it as `temperature_sent`. + - Request profiles reach `triggerRun`/`trigger_run`, `triggerDeliver`/ + `trigger_deliver` and `areev eval`. The bindings read the + `$AREEV_RUN_LLM_*` variables when given no profile, and only when a model + is configured, so a stale variable cannot fail a model-less run. `off` + and `no` mean off, and a bare `--llm-no-temperature` no longer swallows + the next argument. **Behaviour change:** the space form + `--llm-no-temperature false` therefore no longer means off. `false` is + now a positional argument; write `--llm-no-temperature=false`. +- **Schema annotations no longer refuse a tool** (#397). A `tools_from` + entry's `input_schema` (or a `response.json_schema`, or a Definition's + `input_schema`/`output_schema` at write time) that carried `$schema`, + `$id`, `$comment`, `deprecated`, `readOnly`, `writeOnly` or `x-*` was + refused (`RUN-E019`, "unsupported keyword $schema"). MCP servers publish + these routinely. They are now accepted and dropped at every level + (`json_schema_subset::strip_annotations`). `title`, `description`, + `examples` and `default` were already accepted and are kept, and the + structural subset rule is unchanged. +- **A streamed turn records `served_model`** (#394). The OpenAI-compatible, + Anthropic and Ollama stream accumulators now read the served model as the + whole-response parsers do. A run observed through `onEvent`, which streams, + no longer journals less provenance than the same run unobserved. +- **An abstract node's call to a code-carrying Definition runs its blob.** The + driver looked up a model-issued call's code by node, found the abstract + node's own pin, and sent the call to `--tool-cmd`. It now finds the + Definition by its frozen content address. +- **The CLI's `--llm-token-field`, `--llm-no-temperature` and + `--llm-extra-body` take effect.** `areev run`'s usage text has listed them + since #285, but nothing read them. They now build the request profile + (`$AREEV_RUN_LLM_*` twins). + ## [1.12.2] — 2026-10-06 ### Fixed @@ -4794,7 +4906,8 @@ ecosystem adapters, and the enterprise plane. `crates/areev-bench` (`RESULTS.md` has the numbers), with perf gates (`bench`, `voice_loop`) run as examples. -[Unreleased]: https://github.com/AreevAI/areev/compare/v1.12.2...HEAD +[Unreleased]: https://github.com/AreevAI/areev/compare/v1.13.0...HEAD +[1.13.0]: https://github.com/AreevAI/areev/compare/v1.12.2...v1.13.0 [1.12.2]: https://github.com/AreevAI/areev/compare/v1.12.1...v1.12.2 [1.12.1]: https://github.com/AreevAI/areev/compare/v1.12.0...v1.12.1 [1.12.0]: https://github.com/AreevAI/areev/compare/v1.11.1...v1.12.0 diff --git a/Cargo.lock b/Cargo.lock index 885ce8cc..ac4ec43b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -151,7 +151,7 @@ dependencies = [ [[package]] name = "areev" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-context", @@ -177,7 +177,7 @@ dependencies = [ [[package]] name = "areev-bench" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-context", @@ -195,7 +195,7 @@ dependencies = [ [[package]] name = "areev-cal" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-core", "areev-store", @@ -217,7 +217,7 @@ dependencies = [ [[package]] name = "areev-conformance" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev", "areev-cal", @@ -233,7 +233,7 @@ dependencies = [ [[package]] name = "areev-context" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -246,7 +246,7 @@ dependencies = [ [[package]] name = "areev-core" -version = "1.12.2" +version = "1.13.0" dependencies = [ "hex", "insta", @@ -262,7 +262,7 @@ dependencies = [ [[package]] name = "areev-llm" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-core", "areev-loop", @@ -274,7 +274,7 @@ dependencies = [ [[package]] name = "areev-loop" -version = "1.12.2" +version = "1.13.0" dependencies = [ "serde", "serde_json", @@ -282,7 +282,7 @@ dependencies = [ [[package]] name = "areev-loop-adapter" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -297,7 +297,7 @@ dependencies = [ [[package]] name = "areev-mcp" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -312,7 +312,7 @@ dependencies = [ [[package]] name = "areev-py" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev", "areev-cal", @@ -329,7 +329,7 @@ dependencies = [ [[package]] name = "areev-run" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -347,7 +347,7 @@ dependencies = [ [[package]] name = "areev-run-core" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-core", "serde", @@ -357,7 +357,7 @@ dependencies = [ [[package]] name = "areev-server" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -379,7 +379,7 @@ dependencies = [ [[package]] name = "areev-store" -version = "1.12.2" +version = "1.13.0" dependencies = [ "aes-gcm 0.11.1", "areev-core", @@ -405,7 +405,7 @@ dependencies = [ [[package]] name = "areev-trigger" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -3200,9 +3200,9 @@ dependencies = [ [[package]] name = "rustls" -version = "0.23.45" +version = "0.23.46" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634" +checksum = "48e13bd8c0e9365c43cfa5c9e8f9ad49d3c8444926c9aac819e0e4dc503c8fdf" dependencies = [ "log", "once_cell", diff --git a/Cargo.toml b/Cargo.toml index 5b25d4fa..c9eeeb91 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -12,7 +12,7 @@ exclude = ["areev-sandbox", "areev-tools"] members = ["crates/areev-core", "crates/areev-store", "crates/areev-cal", "crates/areev-cli", "crates/areev-mcp", "crates/areev-context", "crates/areev-server", "crates/areev-py", "crates/areev-bench", "crates/areev-loop", "crates/areev-loop-adapter", "crates/areev-llm", "crates/areev-conformance", "crates/areev-run-core", "crates/areev-run", "crates/areev-trigger"] [workspace.package] -version = "1.12.2" +version = "1.13.0" edition = "2021" license = "MIT OR Apache-2.0" authors = ["MindGryd Software Private Limited"] diff --git a/ERROR_CODES.md b/ERROR_CODES.md index b85164d4..fa1b7e9f 100644 --- a/ERROR_CODES.md +++ b/ERROR_CODES.md @@ -174,6 +174,7 @@ in source. | `RUN-E029` | `NotPausable` | A pause was asked of a run that cannot be paused (#344): it already finished — completed, failed, stalled, canceled or out of budget — or a cancel is pending against it (cancel wins over pause; `cancel` on a paused run finalizes it). Pausing an already-paused or already-pause-requested run is NOT this error: it is idempotent and answers with the standing request | | `RUN-E030` | `NoDecider` | A plan binds a decision node (a Tool Definition whose `executor_uri` is `areev://decide`) and this host has no decision backend installed. Raised at run start, the V7 freeze, before the run exists and naming the node. `resume` and a forking `fork` raise it too, before the lease is taken, on a host without a backend. Install one (`--decide ` / `--decide-cmd`, `$AREEV_DECIDE`, `Runner::with_decider`) or bind the node to an ordinary tool. The scheduler's OPTIONAL asks (the decision-guided fold and the tool-offer narrowing) never raise it: without a backend they are not asked | | `RUN-E031` | `SavedQueryUnavailable` | A plan's `op: query` read (#370) cannot be served. At run start (the V7 freeze, before the run exists, naming the node): no saved query has the name, the declaration supplies a parameter the query does not declare, or it leaves a required parameter unsupplied. At dispatch (the node fails, not retried): the body pinned into the manifest at start is missing, or no longer matches the SHA-256 recorded beside it. The run reads the PINNED body, never the live `qry:` row, so a query redefined or dropped mid-run is not this error — it cannot reach a running run at all | +| `RUN-W001` | (warning, `RunManifest.warnings`) | A node binds a client or decision Definition that declares `locked_params` (#393). No model calls such a tool, so there is nothing to lock. The locks are ignored, as 1.12 ignored them, and the start records this warning, which `run inspect` shows under `warnings`. The run is not refused | ### `TRG` — triggers (`areev-trigger/src/error.rs`) diff --git a/areev-sandbox/Cargo.lock b/areev-sandbox/Cargo.lock index 0c01561a..cec2a5fe 100644 --- a/areev-sandbox/Cargo.lock +++ b/areev-sandbox/Cargo.lock @@ -4,7 +4,7 @@ version = 4 [[package]] name = "areev-sandbox" -version = "1.12.2" +version = "1.13.0" dependencies = [ "serde", "serde_json", diff --git a/areev-sandbox/Cargo.toml b/areev-sandbox/Cargo.toml index 277b31cc..f31e2c39 100644 --- a/areev-sandbox/Cargo.toml +++ b/areev-sandbox/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "areev-sandbox" -version = "1.12.2" +version = "1.13.0" edition = "2021" # wasmi's stable line sets this; deliberately NOT inherited from the workspace, # because this package is not a member of it. diff --git a/crates/areev-bench/Cargo.toml b/crates/areev-bench/Cargo.toml index b63fc436..2d9bf19e 100644 --- a/crates/areev-bench/Cargo.toml +++ b/crates/areev-bench/Cargo.toml @@ -11,14 +11,14 @@ homepage.workspace = true publish = false [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } # `cargo run -p areev-bench --features postgres --bin pg_bench` — the # server-tier latency harness (needs DATABASE_URL). -areev-store = { path = "../areev-store", version = "1.12.0" } -areev-cal = { path = "../areev-cal", version = "1.12.0" } -areev-context = { path = "../areev-context", version = "1.12.0" } -areev-server = { path = "../areev-server", version = "1.12.0" } +areev-store = { path = "../areev-store", version = "1.13.0" } +areev-cal = { path = "../areev-cal", version = "1.13.0" } +areev-context = { path = "../areev-context", version = "1.13.0" } +areev-server = { path = "../areev-server", version = "1.13.0" } areev-loop = { path = "../areev-loop" } areev-loop-adapter = { path = "../areev-loop-adapter" } areev-llm = { path = "../areev-llm" } diff --git a/crates/areev-cal/Cargo.toml b/crates/areev-cal/Cargo.toml index 7c6056eb..81d9040b 100644 --- a/crates/areev-cal/Cargo.toml +++ b/crates/areev-cal/Cargo.toml @@ -13,8 +13,8 @@ keywords = ["query-language", "parser", "memory", "cal", "ai-agents"] categories = ["parser-implementations", "database-implementations"] [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } -areev-store = { path = "../areev-store", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } +areev-store = { path = "../areev-store", version = "1.13.0" } serde = { version = "1", features = ["derive"] } serde_json = "1" logos = "0.14" diff --git a/crates/areev-cal/src/json_build.rs b/crates/areev-cal/src/json_build.rs index 60007540..c541c4ac 100644 --- a/crates/areev-cal/src/json_build.rs +++ b/crates/areev-cal/src/json_build.rs @@ -1112,11 +1112,17 @@ pub fn build_grain_from_json( if let Some(p) = strict_str("parent_task_id")? { tool = tool.parent_task_id(&p); } + // Annotation keywords (`$schema`, `$id`, `$comment`, + // `deprecated`, `readOnly`, `writeOnly`, `x-*`) are accepted + // and dropped before the subset check (#397): they change no + // validation, and the stored schema is the clean form. if let Some(schema) = fields.get("output_schema").filter(|v| !v.is_null()) { - SchemaValidator::tool_schema().validate(schema).map_err(|e| { + let mut schema = schema.clone(); + areev_core::types::json_schema_subset::strip_annotations(&mut schema); + SchemaValidator::tool_schema().validate(&schema).map_err(|e| { AreevError::Validation(format!("tool 'output_schema' rejected: {e}")) })?; - tool = tool.output_schema(schema.clone()); + tool = tool.output_schema(schema); } // ── Definition vocabulary + async lifecycle (see the // `type_known_fields` comment). Enum strings parse strictly: @@ -1136,10 +1142,12 @@ pub fn build_grain_from_json( tool = tool.tool_description(&d); } if let Some(schema) = fields.get("input_schema").filter(|v| !v.is_null()) { - SchemaValidator::tool_schema().validate(schema).map_err(|e| { + let mut schema = schema.clone(); + areev_core::types::json_schema_subset::strip_annotations(&mut schema); + SchemaValidator::tool_schema().validate(&schema).map_err(|e| { AreevError::Validation(format!("tool 'input_schema' rejected: {e}")) })?; - tool = tool.input_schema(schema.clone()); + tool = tool.input_schema(schema); } if let Some(s) = strict_bool("strict")? { tool = tool.strict(s); @@ -1649,6 +1657,43 @@ mod tests { ); } + /// #397: a Definition's schema carrying the annotations MCP servers + /// publish is accepted, stored without them; the subset rule holds. + #[test] + fn a_definitions_schema_annotations_are_accepted_and_dropped() { + struct Capture; + impl GrainSink for Capture { + type Out = Option; + fn consume(self, grain: &G) -> Result { + let any = grain as &dyn std::any::Any; + Ok(any.downcast_ref::().and_then(|t| t.input_schema.clone())) + } + } + let fields = |schema: serde_json::Value| { + let mut f = serde_json::Map::new(); + f.insert("tool_name".into(), json!("lookup")); + f.insert("kind".into(), json!("definition")); + f.insert("input_schema".into(), schema); + f + }; + let stored = build_grain_from_json( + "tool", + &fields(json!({"$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "urn:l", + "type": "object", "deprecated": false, + "properties": {"q": {"type": "string", "readOnly": true}}})), + Capture, + ) + .unwrap() + .unwrap(); + assert_eq!(stored, json!({"type": "object", "properties": {"q": {"type": "string"}}})); + assert!(build_grain_from_json( + "tool", + &fields(json!({"$schema": "x", "type": "object", "oneOf": []})), + NullSink + ) + .is_err()); + } + #[test] fn event_json_reaches_every_typed_slot() { let fields = json!({ diff --git a/crates/areev-cli/Cargo.toml b/crates/areev-cli/Cargo.toml index c525b469..299e2730 100644 --- a/crates/areev-cli/Cargo.toml +++ b/crates/areev-cli/Cargo.toml @@ -41,9 +41,9 @@ tls = ["areev-server/tls"] oidc = ["areev-server/oidc"] [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } -areev-store = { path = "../areev-store", version = "1.12.0" } -areev-cal = { path = "../areev-cal", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } +areev-store = { path = "../areev-store", version = "1.13.0" } +areev-cal = { path = "../areev-cal", version = "1.13.0" } # Already in the tree via every other crate; adds no new package. serde = { version = "1", features = ["derive"] } serde_json = "1" @@ -57,15 +57,15 @@ getrandom = "0.4" # ISO-8601 transcript timestamps so capture-stop can pin created_at. chrono = { version = "0.4", default-features = false, features = ["std"] } -areev-mcp = { path = "../areev-mcp", version = "1.12.0" } -areev-context = { path = "../areev-context", version = "1.12.0" } -areev-server = { path = "../areev-server", version = "1.12.0" } -areev-loop = { path = "../areev-loop", version = "1.12.0" } -areev-loop-adapter = { path = "../areev-loop-adapter", version = "1.12.0" } -areev-llm = { path = "../areev-llm", version = "1.12.0" } -areev-run = { path = "../areev-run", version = "1.12.0" } -areev-trigger = { path = "../areev-trigger", version = "1.12.0", features = ["tz"] } -areev-run-core = { path = "../areev-run-core", version = "1.12.0" } +areev-mcp = { path = "../areev-mcp", version = "1.13.0" } +areev-context = { path = "../areev-context", version = "1.13.0" } +areev-server = { path = "../areev-server", version = "1.13.0" } +areev-loop = { path = "../areev-loop", version = "1.13.0" } +areev-loop-adapter = { path = "../areev-loop-adapter", version = "1.13.0" } +areev-llm = { path = "../areev-llm", version = "1.13.0" } +areev-run = { path = "../areev-run", version = "1.13.0" } +areev-trigger = { path = "../areev-trigger", version = "1.13.0", features = ["tz"] } +areev-run-core = { path = "../areev-run-core", version = "1.13.0" } [dev-dependencies] tempfile = "3" diff --git a/crates/areev-cli/src/main.rs b/crates/areev-cli/src/main.rs index caae9e9d..9c67c530 100644 --- a/crates/areev-cli/src/main.rs +++ b/crates/areev-cli/src/main.rs @@ -1385,12 +1385,31 @@ fn addr_is_loopback(addr: &str) -> bool { } } +/// Switches that never take a value: a bare `--llm-no-temperature` followed by +/// a positional must not swallow it as its value. `--switch=false` still +/// spells it off. +const BARE_SWITCHES: &[&str] = &["llm-no-temperature"]; + fn parse_args(rest: &[String]) -> (HashMap, Vec) { let mut flags = HashMap::new(); let mut positional = Vec::new(); let mut i = 0; while i < rest.len() { let a = &rest[i]; + if let Some(name) = a.strip_prefix("--").filter(|n| BARE_SWITCHES.contains(n)) { + flags.insert(name.to_string(), "true".to_string()); + i += 1; + continue; + } + if let Some((name, value)) = a + .strip_prefix("--") + .and_then(|n| n.split_once('=')) + .filter(|(n, _)| BARE_SWITCHES.contains(n)) + { + flags.insert(name.to_string(), value.to_string()); + i += 1; + continue; + } if let Some(name) = a.strip_prefix("--") { // A value is the next token only if it is not itself a flag. Reject // any leading '-' (not just "--"), so a valueless long flag doesn't @@ -5362,10 +5381,14 @@ fn run_eval( (None, None) => return Err(usage.to_string()), (Some(_), None) => None, (None, Some(spec)) => Some( - areev_llm::resolve_toolcall( + // The same request profile `areev run` builds (#394): + // grading a model that refuses a temperature must not + // 400 where running it does not. + areev_llm::resolve_toolcall_with_profile( spec, flag(flags, "base-url").as_deref(), flag(flags, "key-env").as_deref(), + run_stack::request_profile(flags)?, ) .map_err(|e| e.to_string())?, ), @@ -5919,6 +5942,7 @@ fn run_eval_case_model( tool_choice: ToolChoice::None, max_tokens, temperature: 0.0, + ..Default::default() }; match llm.call(&req) { Ok(resp) => { @@ -7722,6 +7746,18 @@ mod tests { parse_args(&a.iter().map(|s| s.to_string()).collect::>()) } + // ── a bare switch takes no value (#394 review) ───────────────────── + + #[test] + fn a_bare_switch_never_swallows_the_next_positional() { + let (f, p) = args(&["--llm-no-temperature", "start", "--model", "m"]); + assert_eq!(f.get("llm-no-temperature").map(String::as_str), Some("true")); + assert_eq!(p, vec!["start".to_string()]); + assert_eq!(f.get("model").map(String::as_str), Some("m")); + let (f, _) = args(&["--llm-no-temperature=off"]); + assert_eq!(f.get("llm-no-temperature").map(String::as_str), Some("off")); + } + // ── --mount parsing (issue #184) ──────────────────────────────────── #[test] diff --git a/crates/areev-cli/src/run_stack.rs b/crates/areev-cli/src/run_stack.rs index d16ff95e..100d364d 100644 --- a/crates/areev-cli/src/run_stack.rs +++ b/crates/areev-cli/src/run_stack.rs @@ -302,18 +302,58 @@ pub fn can_execute(flags: &HashMap) -> bool { pub fn toolcall_llm( flags: &HashMap, ) -> Result>, String> { + let profile = request_profile(flags)?; match flag_or_env(flags, "model", "AREEV_RUN_MODEL") { None => Ok(None), - Some(spec) => areev_llm::resolve_toolcall( + Some(spec) => areev_llm::resolve_toolcall_with_profile( &spec, flag_or_env(flags, "base-url", "AREEV_RUN_BASE_URL").as_deref(), flag_or_env(flags, "key-env", "AREEV_RUN_KEY_ENV").as_deref(), + profile, ) .map(Some) .map_err(|e| e.to_string()), } } +/// The endpoint's request profile (#285, #394): `--llm-token-field +/// max_tokens|max_completion_tokens`, `--llm-no-temperature`, `--llm-extra-body +/// JSON`, each with an `$AREEV_RUN_LLM_*` twin. `None` when none is given. +/// +/// The USAGE text named these flags for a release in which nothing read them; +/// they are parsed here, through the same `RequestProfile` the bindings' +/// `llm_profile` builds, so a misconfiguration is refused before a run exists. +pub fn request_profile( + flags: &HashMap, +) -> Result, String> { + use areev_llm::profile::{RequestProfile, TemperatureMode, TokenLimitField}; + let mut p = RequestProfile::new(); + let mut any = false; + if let Some(f) = flag_or_env(flags, "llm-token-field", "AREEV_RUN_LLM_TOKEN_FIELD") { + any = true; + p = p.token_limit(match f.as_str() { + "max_tokens" => TokenLimitField::MaxTokens, + "max_completion_tokens" => TokenLimitField::MaxCompletionTokens, + other => { + return Err(format!( + "--llm-token-field {other:?}: expected max_tokens or max_completion_tokens" + )) + } + }); + } + if flag_or_env(flags, "llm-no-temperature", "AREEV_RUN_LLM_NO_TEMPERATURE") + .is_some_and(|v| areev_llm::profile::is_truthy(&v)) + { + any = true; + p = p.temperature(TemperatureMode::Omit); + } + if let Some(body) = flag_or_env(flags, "llm-extra-body", "AREEV_RUN_LLM_EXTRA_BODY") { + any = true; + p = p.extra_body_json(&body).map_err(|e| e.to_string())?; + } + Ok(any.then_some(p)) +} + /// `--events` prints each §6.10 run event to stderr as one JSON line (stdout /// stays the machine surface); `--otel-endpoint` exports OTLP/HTTP JSON spans /// to a collector. Both compose through one fan-out observer. @@ -769,4 +809,29 @@ mod tests { ExecResult::Err { detail, .. } => panic!("{detail}"), } } + #[test] + fn the_llm_profile_flags_build_a_profile_and_refuse_a_bad_one() { + let _g = env_guard(); + let flags = |pairs: &[(&str, &str)]| -> HashMap { + pairs.iter().map(|(k, v)| (k.to_string(), v.to_string())).collect() + }; + assert!(request_profile(&flags(&[])).unwrap().is_none(), "no flag, no profile"); + let p = request_profile(&flags(&[ + ("llm-token-field", "max_completion_tokens"), + ("llm-no-temperature", "true"), + ("llm-extra-body", r#"{"reasoning_effort":"low"}"#), + ])) + .unwrap() + .unwrap(); + assert_eq!(p.token_limit_key(), "max_completion_tokens"); + assert!(!p.sends_temperature()); + assert!(request_profile(&flags(&[("llm-token-field", "tokens")])).is_err()); + for off in ["off", "no", "0", "false"] { + assert!( + request_profile(&flags(&[("llm-no-temperature", off)])).unwrap().is_none(), + "{off} is off" + ); + } + assert!(request_profile(&flags(&[("llm-extra-body", r#"{"model":"x"}"#)])).is_err()); + } } diff --git a/crates/areev-context/Cargo.toml b/crates/areev-context/Cargo.toml index bf0b8241..e6fcfd26 100644 --- a/crates/areev-context/Cargo.toml +++ b/crates/areev-context/Cargo.toml @@ -20,8 +20,8 @@ llm-rerank = [] rerank = [] [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } -areev-cal = { path = "../areev-cal", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } +areev-cal = { path = "../areev-cal", version = "1.13.0" } serde = { version = "1", features = ["derive"] } serde_json = "1" chrono = { version = "0.4", default-features = false, features = ["clock"] } diff --git a/crates/areev-core/src/types/json_schema_subset.rs b/crates/areev-core/src/types/json_schema_subset.rs index 9b961703..9a830d44 100644 --- a/crates/areev-core/src/types/json_schema_subset.rs +++ b/crates/areev-core/src/types/json_schema_subset.rs @@ -208,6 +208,54 @@ fn strip_vendor_extensions_at(schema: &mut Value, depth: usize) { } } +/// JSON Schema annotation keywords that change no validation outcome and are +/// not in [`ALLOWED_KEYWORDS`] (#397). MCP servers publish tool schemas +/// carrying them routinely (`$schema` above all), and refusing a schema for an +/// annotation refuses a tool for its paperwork. [`strip_annotations`] removes +/// them; `title`, `description`, `examples` and `default` are annotations +/// too, but already in the portable subset, so they stay. +pub const ANNOTATION_KEYWORDS: &[&str] = + &["$schema", "$id", "$comment", "deprecated", "readOnly", "writeOnly"]; + +/// Remove [`ANNOTATION_KEYWORDS`] and vendor extensions (`x-*`) at every +/// level of a schema, in place — accept and ignore them, rather than refuse +/// the schema (#397). Structure-aware exactly as +/// [`strip_vendor_extensions`]: only keyword positions are touched, so a +/// PROPERTY named `title` or `$id` survives. Run before +/// [`SchemaValidator::validate`]; the structural subset rule (no `$ref`, +/// `oneOf`, `anyOf`, `allOf`, `patternProperties`, …) is unchanged. +pub fn strip_annotations(schema: &mut Value) { + strip_keywords_at(schema, 0, &|k: &str| k.starts_with("x-") || ANNOTATION_KEYWORDS.contains(&k)); +} + +fn strip_keywords_at(schema: &mut Value, depth: usize, drop: &dyn Fn(&str) -> bool) { + if depth > MAX_SCHEMA_DEPTH { + return; + } + match schema { + Value::Object(map) => { + map.retain(|k, _| !drop(k)); + if let Some(Value::Object(props)) = map.get_mut("properties") { + for sub in props.values_mut() { + strip_keywords_at(sub, depth + 1, drop); + } + } + if let Some(items) = map.get_mut("items") { + strip_keywords_at(items, depth + 1, drop); + } + if let Some(addl) = map.get_mut("additionalProperties") { + strip_keywords_at(addl, depth + 1, drop); + } + } + Value::Array(items) => { + for v in items.iter_mut() { + strip_keywords_at(v, depth + 1, drop); + } + } + _ => {} + } +} + /// Validator for the tool-schema subset. Stateless; cheap to construct. pub struct SchemaValidator { allowed_keywords: &'static [&'static str], @@ -811,6 +859,35 @@ mod tests { assert_eq!(err, InstanceErrorKind::Shape); } + // ── strip_annotations (#397) ── + + #[test] + fn annotation_keywords_are_stripped_at_every_level_and_property_names_survive() { + let mut s = json!({ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "urn:t", "$comment": "c", "title": "T", "description": "d", + "type": "object", + "properties": { + "$id": {"type": "string", "readOnly": true, "deprecated": true}, + "list": {"type": "array", "items": {"type": "string", "writeOnly": true, + "examples": ["a"], "default": "a"}}, + "map": {"type": "object", "additionalProperties": {"type": "integer", "$comment": "n"}} + } + }); + strip_annotations(&mut s); + assert!(SchemaValidator::tool_schema().validate(&s).is_ok(), "{s}"); + assert!(s.get("$schema").is_none() && s.get("$id").is_none() && s.get("$comment").is_none()); + assert_eq!(s["title"], "T", "already portable: kept"); + assert!(s["properties"].get("$id").is_some(), "a property NAMED $id survives"); + assert!(s["properties"]["$id"].get("readOnly").is_none()); + assert!(s["properties"]["list"]["items"].get("writeOnly").is_none()); + assert!(s["properties"]["map"]["additionalProperties"].get("$comment").is_none()); + // The structural rule did not move. + let mut banned = json!({"type": "object", "$schema": "x", "anyOf": []}); + strip_annotations(&mut banned); + assert!(SchemaValidator::tool_schema().validate(&banned).is_err()); + } + // ── strip_vendor_extensions ── fn no_vendor_keys(v: &Value) -> bool { diff --git a/crates/areev-js/Cargo.lock b/crates/areev-js/Cargo.lock index ed181875..a2d12300 100644 --- a/crates/areev-js/Cargo.lock +++ b/crates/areev-js/Cargo.lock @@ -151,7 +151,7 @@ dependencies = [ [[package]] name = "areev" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-context", @@ -176,7 +176,7 @@ dependencies = [ [[package]] name = "areev-cal" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-core", "areev-store", @@ -197,7 +197,7 @@ dependencies = [ [[package]] name = "areev-context" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -208,7 +208,7 @@ dependencies = [ [[package]] name = "areev-core" -version = "1.12.2" +version = "1.13.0" dependencies = [ "hex", "regex", @@ -222,7 +222,7 @@ dependencies = [ [[package]] name = "areev-js" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev", "areev-cal", @@ -241,7 +241,7 @@ dependencies = [ [[package]] name = "areev-llm" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-core", "areev-loop", @@ -252,7 +252,7 @@ dependencies = [ [[package]] name = "areev-loop" -version = "1.12.2" +version = "1.13.0" dependencies = [ "serde", "serde_json", @@ -260,7 +260,7 @@ dependencies = [ [[package]] name = "areev-loop-adapter" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -274,7 +274,7 @@ dependencies = [ [[package]] name = "areev-mcp" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -288,7 +288,7 @@ dependencies = [ [[package]] name = "areev-run" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -305,7 +305,7 @@ dependencies = [ [[package]] name = "areev-run-core" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-core", "serde", @@ -315,7 +315,7 @@ dependencies = [ [[package]] name = "areev-server" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -328,7 +328,7 @@ dependencies = [ [[package]] name = "areev-store" -version = "1.12.2" +version = "1.13.0" dependencies = [ "aes-gcm 0.11.1", "areev-core", @@ -352,7 +352,7 @@ dependencies = [ [[package]] name = "areev-trigger" -version = "1.12.2" +version = "1.13.0" dependencies = [ "areev-cal", "areev-core", @@ -2786,9 +2786,9 @@ dependencies = [ [[package]] name = "rustls" -version = "0.23.45" +version = "0.23.46" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634" +checksum = "48e13bd8c0e9365c43cfa5c9e8f9ad49d3c8444926c9aac819e0e4dc503c8fdf" dependencies = [ "log", "once_cell", diff --git a/crates/areev-js/Cargo.toml b/crates/areev-js/Cargo.toml index a84d9d8c..a30e4e1e 100644 --- a/crates/areev-js/Cargo.toml +++ b/crates/areev-js/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "areev-js" -version = "1.12.2" +version = "1.13.0" edition = "2021" license = "MIT OR Apache-2.0" authors = ["MindGryd Software Private Limited"] @@ -21,15 +21,15 @@ publish = false crate-type = ["cdylib"] [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } # The postgres backend ships ON in the binding: it is a server-side surface, # and `new Areev("postgres://…?schema=")` must work from a stock # `npm install @areev/areev`. The dependency-light default stays with the CLI/store. # `postgres-tls` rides with it: a managed Postgres requires TLS on the wire and # an addon installer cannot recompile to add a cargo feature. rustls is already # in this crate's graph via areev-llm's HTTP client. -areev-store = { path = "../areev-store", version = "1.12.0", features = ["postgres", "postgres-tls"] } -areev-cal = { path = "../areev-cal", version = "1.12.0" } +areev-store = { path = "../areev-store", version = "1.13.0", features = ["postgres", "postgres-tls"] } +areev-cal = { path = "../areev-cal", version = "1.13.0" } areev-loop = { path = "../areev-loop" } areev-loop-adapter = { path = "../areev-loop-adapter" } areev-llm = { path = "../areev-llm" } diff --git a/crates/areev-js/__test__/smoke.mjs b/crates/areev-js/__test__/smoke.mjs index 390e8eb3..4abdfab3 100644 --- a/crates/areev-js/__test__/smoke.mjs +++ b/crates/areev-js/__test__/smoke.mjs @@ -1623,6 +1623,61 @@ test('runStart resolves the tool-calling model before journaling a run', async ( m.close() }) +// ── #394: the request profile is the LAST positional argument of runStart +// and runResume, so every existing positional caller keeps its meaning. + +test('runStart and runResume take llmProfile last and refuse a malformed one', async () => { + const dir = mkdtempSync(join(tmpdir(), 'areev-profile-')) + const m = new Areev(join(dir, 'm.db'), 'caller') + const startArgs = (profile) => [ + 'e'.repeat(HEX64), 'r1', null, null, null, null, null, null, + 'ollama:llama3.1', null, null, null, null, null, null, null, null, null, + null, null, null, null, null, null, null, null, profile, + ] + await assert.rejects(() => m.runStart(...startArgs('{"temprature":"omit"}')), /temprature/) + await assert.rejects(() => m.runStart(...startArgs('not json')), /llm_profile/) + // A profile the ollama transport cannot honour is refused, not ignored. + await assert.rejects(() => m.runStart(...startArgs('{"temperature":"omit"}')), /no request profile/) + await assert.rejects( + () => m.runResume('r1', null, 'ollama:llama3.1', null, null, null, null, null, null, null, + null, null, null, null, null, null, null, '{"token_field":"tokens"}'), + /token_field/, + ) + assert.equal(JSON.parse(await m.runList(10)).length, 0, 'no run was journaled') + m.close() +}) + +// #394 review: the trigger paths take the profile too, and with no +// argument the bindings read the CLI's $AREEV_RUN_LLM_* twins. +test('triggerRun takes llmProfile last, and the env twins stand in for a missing one', async () => { + const dir = mkdtempSync(join(tmpdir(), 'areev-profile-env-')) + const m = new Areev(join(dir, 'm.db'), 'caller') + const trig = new Array(30).fill(null) + trig[8] = 'ollama:llama3.1' + trig[29] = '{"temprature":"omit"}' + await assert.rejects(() => m.triggerRun(...trig), /temprature/) + const startArgs = [ + 'e'.repeat(HEX64), 'r1', null, null, null, null, null, null, 'ollama:llama3.1', + ] + process.env.AREEV_RUN_LLM_NO_TEMPERATURE = '1' + try { + await assert.rejects(() => m.runStart(...startArgs), /no request profile/) + process.env.AREEV_RUN_LLM_NO_TEMPERATURE = 'off' + // `off` is off: no profile, so the start proceeds to the (absent) plan. + await assert.rejects(() => m.runStart(...startArgs), (e) => !/no request profile/.test(String(e))) + } finally { + delete process.env.AREEV_RUN_LLM_NO_TEMPERATURE + } + // A stale, malformed twin cannot fail a run that calls no model. + process.env.AREEV_RUN_LLM_TOKEN_FIELD = 'tokens' + try { + await assert.rejects(() => m.runStart('e'.repeat(HEX64), 'r2'), (e) => !/AREEV_RUN_LLM/.test(String(e))) + } finally { + delete process.env.AREEV_RUN_LLM_TOKEN_FIELD + } + m.close() +}) + // ── #67: the generic authoring path must validate too. `triggerAdd` alone was // not enough — `add("trigger", …)` is what a host actually reaches for, and it // stored declarations that could never fire. diff --git a/crates/areev-js/index.d.ts b/crates/areev-js/index.d.ts index 82d21915..d788cb2d 100644 --- a/crates/areev-js/index.d.ts +++ b/crates/areev-js/index.d.ts @@ -672,8 +672,18 @@ export declare class Areev { * Note `RunFinished` is emitted at a TERMINAL outcome: a run that parks * on a human gate ends this leg at `AskRaised`, and the `runResume` leg * carries `RunResumed` … `RunFinished`. + * + * `llmProfile` (#394, the LAST argument) shapes every request the model + * transport sends, as a JSON string: `{"temperature": "omit" | , + * "token_field": "max_tokens" | "max_completion_tokens", "extra_body": + * {…}}`, each key optional — the CLI's `--llm-no-temperature` / + * `--llm-token-field` / `--llm-extra-body`. `'{"temperature":"omit"}'` is + * what runs the GPT-5 family, which answers 400 to any temperature. An + * unknown key is refused. The run pins the profile's digest, so + * `runResume` must be given the same one. A plan's per-node `llm` block + * (`docs/run.md`) configures abstract nodes without it. */ - runStart(workflow: string, runId: string, inputJson?: string | undefined | null, toolCmd?: string | undefined | null, maxTokens?: number | undefined | null, maxUsdMicros?: number | undefined | null, maxWallMs?: number | undefined | null, askTtlSec?: number | undefined | null, model?: string | undefined | null, baseUrl?: string | undefined | null, keyEnv?: string | undefined | null, llmMaxTokens?: number | undefined | null, allowExecutor?: string | undefined | null, executorCache?: string | undefined | null, sandboxCmd?: string | undefined | null, executorTimeoutSecs?: number | undefined | null, toolEnv?: string | undefined | null, onEvent?: ((arg: string) => void) | undefined | null, credentials?: string | undefined | null, allowHosts?: string | undefined | null, toolEgress?: string | undefined | null, credentialTtlSecs?: number | undefined | null, resolverEnv?: string | undefined | null, maxEffectsPerAttempt?: number | undefined | null, llmToolResultChars?: number | undefined | null, llmContextTokens?: number | undefined | null): Promise + runStart(workflow: string, runId: string, inputJson?: string | undefined | null, toolCmd?: string | undefined | null, maxTokens?: number | undefined | null, maxUsdMicros?: number | undefined | null, maxWallMs?: number | undefined | null, askTtlSec?: number | undefined | null, model?: string | undefined | null, baseUrl?: string | undefined | null, keyEnv?: string | undefined | null, llmMaxTokens?: number | undefined | null, allowExecutor?: string | undefined | null, executorCache?: string | undefined | null, sandboxCmd?: string | undefined | null, executorTimeoutSecs?: number | undefined | null, toolEnv?: string | undefined | null, onEvent?: ((arg: string) => void) | undefined | null, credentials?: string | undefined | null, allowHosts?: string | undefined | null, toolEgress?: string | undefined | null, credentialTtlSecs?: number | undefined | null, resolverEnv?: string | undefined | null, maxEffectsPerAttempt?: number | undefined | null, llmToolResultChars?: number | undefined | null, llmContextTokens?: number | undefined | null, llmProfile?: string | undefined | null): Promise /** * Resume a parked/interrupted run from its latest checkpoint. * @@ -684,7 +694,7 @@ export declare class Areev { * including the `RunFinished` a parked start leg never reached — so a * host that watched the start must be able to watch the rest. */ - runResume(runId: string, toolCmd?: string | undefined | null, model?: string | undefined | null, baseUrl?: string | undefined | null, keyEnv?: string | undefined | null, llmMaxTokens?: number | undefined | null, allowExecutor?: string | undefined | null, executorCache?: string | undefined | null, sandboxCmd?: string | undefined | null, executorTimeoutSecs?: number | undefined | null, toolEnv?: string | undefined | null, onEvent?: ((arg: string) => void) | undefined | null, credentials?: string | undefined | null, allowHosts?: string | undefined | null, toolEgress?: string | undefined | null, credentialTtlSecs?: number | undefined | null, resolverEnv?: string | undefined | null): Promise + runResume(runId: string, toolCmd?: string | undefined | null, model?: string | undefined | null, baseUrl?: string | undefined | null, keyEnv?: string | undefined | null, llmMaxTokens?: number | undefined | null, allowExecutor?: string | undefined | null, executorCache?: string | undefined | null, sandboxCmd?: string | undefined | null, executorTimeoutSecs?: number | undefined | null, toolEnv?: string | undefined | null, onEvent?: ((arg: string) => void) | undefined | null, credentials?: string | undefined | null, allowHosts?: string | undefined | null, toolEgress?: string | undefined | null, credentialTtlSecs?: number | undefined | null, resolverEnv?: string | undefined | null, llmProfile?: string | undefined | null): Promise /** * Answer a pending Client ask. `responder` is REQUIRED — approval * separation of duties (responder ≠ triggering principal) is structural. @@ -837,7 +847,7 @@ export declare class Areev { * such a node refuses with `RUN-E018` — the authorization to execute code * must come from the host, never from the file that carries it. */ - triggerRun(only?: string | undefined | null, dryRun?: boolean | undefined | null, leaseSecs?: number | undefined | null, maxItems?: number | undefined | null, connectorCmd?: string | undefined | null, toolCmd?: string | undefined | null, credentialsJson?: string | undefined | null, node?: string | undefined | null, model?: string | undefined | null, baseUrl?: string | undefined | null, keyEnv?: string | undefined | null, maxTokens?: number | undefined | null, maxUsdMicros?: number | undefined | null, maxWallMs?: number | undefined | null, askTtlSec?: number | undefined | null, llmMaxTokens?: number | undefined | null, allowExecutor?: string | undefined | null, executorCache?: string | undefined | null, sandboxCmd?: string | undefined | null, executorTimeoutSecs?: number | undefined | null, toolEnv?: string | undefined | null, credentials?: string | undefined | null, allowHosts?: string | undefined | null, toolEgress?: string | undefined | null, credentialTtlSecs?: number | undefined | null, resolverEnv?: string | undefined | null, maxEffectsPerAttempt?: number | undefined | null, llmToolResultChars?: number | undefined | null, llmContextTokens?: number | undefined | null): Promise + triggerRun(only?: string | undefined | null, dryRun?: boolean | undefined | null, leaseSecs?: number | undefined | null, maxItems?: number | undefined | null, connectorCmd?: string | undefined | null, toolCmd?: string | undefined | null, credentialsJson?: string | undefined | null, node?: string | undefined | null, model?: string | undefined | null, baseUrl?: string | undefined | null, keyEnv?: string | undefined | null, maxTokens?: number | undefined | null, maxUsdMicros?: number | undefined | null, maxWallMs?: number | undefined | null, askTtlSec?: number | undefined | null, llmMaxTokens?: number | undefined | null, allowExecutor?: string | undefined | null, executorCache?: string | undefined | null, sandboxCmd?: string | undefined | null, executorTimeoutSecs?: number | undefined | null, toolEnv?: string | undefined | null, credentials?: string | undefined | null, allowHosts?: string | undefined | null, toolEgress?: string | undefined | null, credentialTtlSecs?: number | undefined | null, resolverEnv?: string | undefined | null, maxEffectsPerAttempt?: number | undefined | null, llmToolResultChars?: number | undefined | null, llmContextTokens?: number | undefined | null, llmProfile?: string | undefined | null): Promise /** * Hand a webhook or manual payload to a trigger. Areev never opens a * port: the host owns the listener and hands the payload over. @@ -845,7 +855,7 @@ export declare class Areev { * Takes the same budgets and the same executor pin as `triggerRun`: a * delivery starts a real run. */ - triggerDeliver(trigger: string, payloadJson: string, connectorCmd?: string | undefined | null, toolCmd?: string | undefined | null, credentialsJson?: string | undefined | null, model?: string | undefined | null, baseUrl?: string | undefined | null, keyEnv?: string | undefined | null, maxTokens?: number | undefined | null, maxUsdMicros?: number | undefined | null, maxWallMs?: number | undefined | null, askTtlSec?: number | undefined | null, llmMaxTokens?: number | undefined | null, allowExecutor?: string | undefined | null, executorCache?: string | undefined | null, sandboxCmd?: string | undefined | null, executorTimeoutSecs?: number | undefined | null, toolEnv?: string | undefined | null, credentials?: string | undefined | null, allowHosts?: string | undefined | null, toolEgress?: string | undefined | null, credentialTtlSecs?: number | undefined | null, resolverEnv?: string | undefined | null, maxEffectsPerAttempt?: number | undefined | null, llmToolResultChars?: number | undefined | null, llmContextTokens?: number | undefined | null): Promise + triggerDeliver(trigger: string, payloadJson: string, connectorCmd?: string | undefined | null, toolCmd?: string | undefined | null, credentialsJson?: string | undefined | null, model?: string | undefined | null, baseUrl?: string | undefined | null, keyEnv?: string | undefined | null, maxTokens?: number | undefined | null, maxUsdMicros?: number | undefined | null, maxWallMs?: number | undefined | null, askTtlSec?: number | undefined | null, llmMaxTokens?: number | undefined | null, allowExecutor?: string | undefined | null, executorCache?: string | undefined | null, sandboxCmd?: string | undefined | null, executorTimeoutSecs?: number | undefined | null, toolEnv?: string | undefined | null, credentials?: string | undefined | null, allowHosts?: string | undefined | null, toolEgress?: string | undefined | null, credentialTtlSecs?: number | undefined | null, resolverEnv?: string | undefined | null, maxEffectsPerAttempt?: number | undefined | null, llmToolResultChars?: number | undefined | null, llmContextTokens?: number | undefined | null, llmProfile?: string | undefined | null): Promise /** Stop a trigger firing without deleting it. Mandatory reason. */ triggerPause(trigger: string, because: string): Promise /** Let a paused trigger fire again. Mandatory reason. */ diff --git a/crates/areev-js/index.js b/crates/areev-js/index.js index 52b2e111..886d656e 100644 --- a/crates/areev-js/index.js +++ b/crates/areev-js/index.js @@ -89,8 +89,8 @@ function requireNative() { try { const binding = require('@areev/areev-android-arm64') const bindingPackageVersion = require('@areev/areev-android-arm64/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -105,8 +105,8 @@ function requireNative() { try { const binding = require('@areev/areev-android-arm-eabi') const bindingPackageVersion = require('@areev/areev-android-arm-eabi/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -126,8 +126,8 @@ function requireNative() { try { const binding = require('@areev/areev-win32-x64-gnu') const bindingPackageVersion = require('@areev/areev-win32-x64-gnu/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -142,8 +142,8 @@ function requireNative() { try { const binding = require('@areev/areev-win32-x64-msvc') const bindingPackageVersion = require('@areev/areev-win32-x64-msvc/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -159,8 +159,8 @@ function requireNative() { try { const binding = require('@areev/areev-win32-ia32-msvc') const bindingPackageVersion = require('@areev/areev-win32-ia32-msvc/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -175,8 +175,8 @@ function requireNative() { try { const binding = require('@areev/areev-win32-arm64-msvc') const bindingPackageVersion = require('@areev/areev-win32-arm64-msvc/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -194,8 +194,8 @@ function requireNative() { try { const binding = require('@areev/areev-darwin-universal') const bindingPackageVersion = require('@areev/areev-darwin-universal/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -210,8 +210,8 @@ function requireNative() { try { const binding = require('@areev/areev-darwin-x64') const bindingPackageVersion = require('@areev/areev-darwin-x64/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -226,8 +226,8 @@ function requireNative() { try { const binding = require('@areev/areev-darwin-arm64') const bindingPackageVersion = require('@areev/areev-darwin-arm64/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -246,8 +246,8 @@ function requireNative() { try { const binding = require('@areev/areev-freebsd-x64') const bindingPackageVersion = require('@areev/areev-freebsd-x64/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -262,8 +262,8 @@ function requireNative() { try { const binding = require('@areev/areev-freebsd-arm64') const bindingPackageVersion = require('@areev/areev-freebsd-arm64/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -283,8 +283,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-x64-musl') const bindingPackageVersion = require('@areev/areev-linux-x64-musl/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -299,8 +299,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-x64-gnu') const bindingPackageVersion = require('@areev/areev-linux-x64-gnu/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -317,8 +317,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-arm64-musl') const bindingPackageVersion = require('@areev/areev-linux-arm64-musl/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -333,8 +333,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-arm64-gnu') const bindingPackageVersion = require('@areev/areev-linux-arm64-gnu/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -351,8 +351,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-arm-musleabihf') const bindingPackageVersion = require('@areev/areev-linux-arm-musleabihf/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -367,8 +367,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-arm-gnueabihf') const bindingPackageVersion = require('@areev/areev-linux-arm-gnueabihf/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -385,8 +385,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-loong64-musl') const bindingPackageVersion = require('@areev/areev-linux-loong64-musl/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -401,8 +401,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-loong64-gnu') const bindingPackageVersion = require('@areev/areev-linux-loong64-gnu/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -419,8 +419,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-riscv64-musl') const bindingPackageVersion = require('@areev/areev-linux-riscv64-musl/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -435,8 +435,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-riscv64-gnu') const bindingPackageVersion = require('@areev/areev-linux-riscv64-gnu/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -452,8 +452,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-ppc64-gnu') const bindingPackageVersion = require('@areev/areev-linux-ppc64-gnu/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -468,8 +468,8 @@ function requireNative() { try { const binding = require('@areev/areev-linux-s390x-gnu') const bindingPackageVersion = require('@areev/areev-linux-s390x-gnu/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -488,8 +488,8 @@ function requireNative() { try { const binding = require('@areev/areev-openharmony-arm64') const bindingPackageVersion = require('@areev/areev-openharmony-arm64/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -504,8 +504,8 @@ function requireNative() { try { const binding = require('@areev/areev-openharmony-x64') const bindingPackageVersion = require('@areev/areev-openharmony-x64/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -520,8 +520,8 @@ function requireNative() { try { const binding = require('@areev/areev-openharmony-arm') const bindingPackageVersion = require('@areev/areev-openharmony-arm/package.json').version - if (bindingPackageVersion !== '1.12.2' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { - throw new Error(`Native binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { + throw new Error(`Native binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } return binding } catch (e) { @@ -661,8 +661,8 @@ if (!nativeBinding || forceWasi) { if (!candidateFailed) { if (process.env.NAPI_RS_ENFORCE_VERSION_CHECK && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') { const bindingPackageVersion = require('@areev/areev-wasm32-wasi/package.json').version - if (bindingPackageVersion !== '1.12.2') { - throw new Error(`WASI binding package version mismatch, expected 1.12.2 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) + if (bindingPackageVersion !== '1.13.0') { + throw new Error(`WASI binding package version mismatch, expected 1.13.0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`) } } wasiBinding = require('@areev/areev-wasm32-wasi') diff --git a/crates/areev-js/package.json b/crates/areev-js/package.json index 8475df92..300bd67b 100644 --- a/crates/areev-js/package.json +++ b/crates/areev-js/package.json @@ -1,6 +1,6 @@ { "name": "@areev/areev", - "version": "1.12.2", + "version": "1.13.0", "description": "Node.js bindings for Areev, the embedded memory engine for AI agents.", "license": "MIT OR Apache-2.0", "author": "MindGryd Software Private Limited", diff --git a/crates/areev-js/src/lib.rs b/crates/areev-js/src/lib.rs index 4d5f999f..5c7dd61a 100644 --- a/crates/areev-js/src/lib.rs +++ b/crates/areev-js/src/lib.rs @@ -95,12 +95,35 @@ fn resolve_toolcall_llm( model: Option, base_url: Option, key_env: Option, + llm_profile: Option, ) -> napi::Result>> { + // The request profile (#394) — `{"temperature": "omit" | n, "token_field": + // …, "extra_body": {…}}` — the CLI's `--llm-*` flags as one JSON string. + // Parsed before the model, and refused by name: a misspelt key that + // configured nothing is the run that 400s on its first turn. + let explicit = llm_profile.as_deref().map(str::trim).filter(|s| !s.is_empty()).is_some(); + let profile = match llm_profile.as_deref().map(str::trim).filter(|s| !s.is_empty()) { + Some(raw) => Some(areev_llm::profile::RequestProfile::from_json(raw).map_err(err)?), + // No argument: the `$AREEV_RUN_LLM_*` twins the CLI reads, so a host + // configured out of band behaves like `areev run` (#394 review) — + // read only when there is a model to shape, so a stale, malformed + // variable cannot fail a run that calls no model at all. + None if model.is_some() => areev_llm::profile::RequestProfile::from_env().map_err(err)?, + None => None, + }; match model { Some(spec) => Ok(Some( - areev_llm::resolve_toolcall(&spec, base_url.as_deref(), key_env.as_deref()) - .map_err(err)?, + areev_llm::resolve_toolcall_with_profile( + &spec, + base_url.as_deref(), + key_env.as_deref(), + profile, + ) + .map_err(err)?, )), + None if explicit => Err(err(AreevError::Validation( + "llmProfile shapes a model's requests — pass `model` with it".into(), + ))), None => Ok(None), } } @@ -3600,6 +3623,16 @@ impl Areev { /// Note `RunFinished` is emitted at a TERMINAL outcome: a run that parks /// on a human gate ends this leg at `AskRaised`, and the `runResume` leg /// carries `RunResumed` … `RunFinished`. + /// + /// `llmProfile` (#394, the LAST argument) shapes every request the model + /// transport sends, as a JSON string: `{"temperature": "omit" | , + /// "token_field": "max_tokens" | "max_completion_tokens", "extra_body": + /// {…}}`, each key optional — the CLI's `--llm-no-temperature` / + /// `--llm-token-field` / `--llm-extra-body`. `'{"temperature":"omit"}'` is + /// what runs the GPT-5 family, which answers 400 to any temperature. An + /// unknown key is refused. The run pins the profile's digest, so + /// `runResume` must be given the same one. A plan's per-node `llm` block + /// (`docs/run.md`) configures abstract nodes without it. #[napi(ts_return_type = "Promise")] #[allow(clippy::too_many_arguments)] // a flat FFI surface; each knob is a distinct scalar pub fn run_start( @@ -3633,6 +3666,7 @@ impl Areev { max_effects_per_attempt: Option, llm_tool_result_chars: Option, llm_context_tokens: Option, + llm_profile: Option, ) -> napi::Result> { // Built HERE, on the JS thread, before the job is queued — see // [`JsEvents`] for why a plain function cannot reach the event bus. @@ -3655,7 +3689,7 @@ impl Areev { let h = areev_core::error::Hash::from_hex(&workflow).map_err(err)?; // Resolved before the run starts, so a bad model spec or a missing // key fails without journaling a run that cannot advance. - let llm = resolve_toolcall_llm(model, base_url, key_env)?; + let llm = resolve_toolcall_llm(model, base_url, key_env, llm_profile)?; let decider = run_decider(installed, &facade, &ns)?; let runner = js_runner_pinned( facade, @@ -3714,6 +3748,10 @@ impl Areev { tool_egress: Option, credential_ttl_secs: Option, resolver_env: Option, + // Appended (#394): the same request profile `runStart` took. A run + // pins the profile's digest (#287), so a resume must pass the same + // one or it is refused RUN-E025 before it takes the lease. + llm_profile: Option, ) -> napi::Result> { let observer = js_observer(on_event)?; let slot = self.facade.clone(); @@ -3726,7 +3764,7 @@ impl Areev { let egress = js_egress_handle(&path, &JsEgressPin { credentials, allow_hosts, tool_egress, credential_ttl_secs, resolver_env, })?; - let llm = resolve_toolcall_llm(model, base_url, key_env)?; + let llm = resolve_toolcall_llm(model, base_url, key_env, llm_profile)?; let decider = run_decider(installed, &facade, &ns)?; let runner = js_runner_pinned( facade, @@ -4256,6 +4294,8 @@ impl Areev { max_effects_per_attempt: Option, llm_tool_result_chars: Option, llm_context_tokens: Option, + // Appended (#394 review): the request profile `runStart` takes. + llm_profile: Option, ) -> napi::bindgen_prelude::AsyncTask { let slot = self.facade.clone(); let path = self.path.clone(); @@ -4265,7 +4305,7 @@ impl Areev { let installed = installed_chain(&self.decide); StringJob::spawn(move || { let facade = take_facade(&slot)?; - let llm = resolve_toolcall_llm(model, base_url, key_env)?; + let llm = resolve_toolcall_llm(model, base_url, key_env, llm_profile)?; let decider = run_decider(installed, &facade, &ns)?; let ev = js_evaluator( facade, @@ -4342,6 +4382,8 @@ impl Areev { max_effects_per_attempt: Option, llm_tool_result_chars: Option, llm_context_tokens: Option, + // Appended (#394 review): the request profile `runStart` takes. + llm_profile: Option, ) -> napi::bindgen_prelude::AsyncTask { let slot = self.facade.clone(); let path = self.path.clone(); @@ -4353,7 +4395,7 @@ impl Areev { let facade = take_facade(&slot)?; let payload: serde_json::Value = serde_json::from_str(&payload_json) .map_err(|e| err(format!("payloadJson is not JSON: {e}")))?; - let llm = resolve_toolcall_llm(model, base_url, key_env)?; + let llm = resolve_toolcall_llm(model, base_url, key_env, llm_profile)?; let decider = run_decider(installed, &facade, &ns)?; let ev = js_evaluator( facade, diff --git a/crates/areev-llm/Cargo.toml b/crates/areev-llm/Cargo.toml index 7da132ed..b2333822 100644 --- a/crates/areev-llm/Cargo.toml +++ b/crates/areev-llm/Cargo.toml @@ -33,11 +33,11 @@ openrouter = ["openai"] vertex = ["openai"] [dependencies] -areev-loop = { path = "../areev-loop", version = "1.12.0" } +areev-loop = { path = "../areev-loop", version = "1.13.0" } # Wave 0 (governed-agents §6.11): the ToolCallLlm seam renders Tool # Definition grains to each provider's native tools format via core's # tool_schema module — workspace-internal, no new external surface. -areev-core = { path = "../areev-core", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } serde = { version = "1", features = ["derive"] } serde_json = "1" # A small BLOCKING HTTP client — matches the sync-over-private-runtime / std diff --git a/crates/areev-llm/src/lib.rs b/crates/areev-llm/src/lib.rs index a8849499..223e06a5 100644 --- a/crates/areev-llm/src/lib.rs +++ b/crates/areev-llm/src/lib.rs @@ -50,8 +50,8 @@ pub use extract::{extract_facts, extract_pipeline, ground_facts, ExtractedFact, pub use llm_detect::LlmDetector; pub use pseudonymize::{PseudonymizingBackend, PseudonymizingDecider}; pub use toolcall::{ - ChatMessage, StopReason, ToolCallError, ToolCallLlm, ToolCallOut, ToolCallRequest, - ToolCallResponse, ToolChoice, Usage, + ChatMessage, ResponseFormat, StopReason, ToolCallError, ToolCallLlm, ToolCallOut, + ToolCallRequest, ToolCallResponse, ToolChoice, Usage, }; use serde_json::{json, Value}; @@ -723,14 +723,51 @@ pub fn resolve_toolcall( base_url: Option<&str>, key_env: Option<&str>, ) -> Result> { + resolve_toolcall_with_profile(spec, base_url, key_env, None) +} + +/// [`resolve_toolcall`] with a request profile (#285) installed on the +/// transport — what the CLI's `--llm-token-field` / `--llm-no-temperature` / +/// `--llm-extra-body` and the bindings' `llm_profile` build (#394). +/// +/// A non-default profile on the Ollama transport is refused rather than +/// ignored: that body takes none of the profile's fields, and a host that +/// asked to omit the temperature must not be told it did. +pub fn resolve_toolcall_with_profile( + spec: &str, + base_url: Option<&str>, + key_env: Option<&str>, + profile: Option, +) -> Result> { + #[cfg_attr( + not(any(feature = "anthropic", feature = "openai", feature = "ollama")), + allow(unused_variables) + )] + let profile = profile.filter(|p| !p.is_default()); #[allow(unreachable_code)] Ok(match build_provider(spec, base_url, key_env)? { #[cfg(feature = "anthropic")] - Provider::Anthropic(p) => std::sync::Arc::new(p), + Provider::Anthropic(p) => std::sync::Arc::new(match profile { + Some(pr) => p.with_request_profile(pr), + None => p, + }), #[cfg(feature = "openai")] - Provider::OpenAi(p) => std::sync::Arc::new(p), + Provider::OpenAi(p) => std::sync::Arc::new(match profile { + Some(pr) => p.with_request_profile(pr), + None => p, + }), #[cfg(feature = "ollama")] - Provider::Ollama(p) => std::sync::Arc::new(p), + Provider::Ollama(p) => { + if profile.is_some() { + return Err(Error::LlmBackend( + "the ollama transport takes no request profile (token field, temperature \ + mode, extra body) — drop the profile, or reach the model through an \ + OpenAI-compatible endpoint" + .into(), + )); + } + std::sync::Arc::new(p) + } }) } @@ -811,6 +848,25 @@ mod tests { std::env::remove_var("ANTHROPIC_API_KEY"); } + #[test] + #[cfg(feature = "ollama")] + fn a_profile_the_ollama_transport_cannot_honour_is_refused_not_ignored() { + use crate::profile::{RequestProfile, TemperatureMode}; + let omit = RequestProfile::new().temperature(TemperatureMode::Omit); + let err = resolve_toolcall_with_profile("ollama:llama3.1", None, None, Some(omit)) + .err() + .expect("refused"); + assert!(err.to_string().contains("no request profile"), "{err}"); + // A default profile is no profile at all. + assert!(resolve_toolcall_with_profile( + "ollama:llama3.1", + None, + None, + Some(RequestProfile::default()) + ) + .is_ok()); + } + /// `post_json` end-to-end over a real loopback socket (regression for the /// ureq 2→3 migration): the request line, Content-Type, custom headers, /// and body must arrive as sent; the JSON response must round-trip; and a diff --git a/crates/areev-llm/src/profile.rs b/crates/areev-llm/src/profile.rs index a37d5af0..8da7aa26 100644 --- a/crates/areev-llm/src/profile.rs +++ b/crates/areev-llm/src/profile.rs @@ -50,6 +50,17 @@ pub struct RequestProfile { pub token_limit: TokenLimitField, pub temperature: TemperatureMode, extra_body: Map, + /// A temperature this endpoint is always sent (#394), in place of the + /// request's own. `None` sends what the request asks for. Meaningless + /// under [`TemperatureMode::Omit`], which sends none at all. + temperature_value: Option, +} + +/// A switch's value as a boolean: everything is ON except the spellings of +/// OFF — `false`, `0`, `off`, `no` (any case). One reader for the CLI flag, +/// its environment twin and the bindings, so `=off` never means on. +pub fn is_truthy(v: &str) -> bool { + !matches!(v.trim().to_ascii_lowercase().as_str(), "false" | "0" | "off" | "no") } /// Keys the adapters own. `extra_body` may not set any of them: a profile @@ -84,6 +95,138 @@ impl RequestProfile { self } + /// Always send this temperature, whatever the request asks for (#394). + /// The profile describes the ENDPOINT, so it wins over a request's value + /// the way [`TemperatureMode::Omit`] already does. + pub fn temperature_value(mut self, t: f64) -> Self { + self.temperature = TemperatureMode::Send; + self.temperature_value = Some(t); + self + } + + /// The temperature to put on the wire for `req`, or `None` to send none: + /// the profile's omission or the request's own (`omit_temperature`, a + /// plan node's `"temperature": "omit"`) — either is enough — else the + /// profile's pinned value, else the request's. + pub fn temperature_for(&self, req: &crate::toolcall::ToolCallRequest<'_>) -> Option { + if !self.sends_temperature() || req.omit_temperature { + return None; + } + Some(self.temperature_value.unwrap_or(req.temperature)) + } + + /// The temperature this profile puts on the wire for a request asking + /// for `requested`: none under [`TemperatureMode::Omit`], else the pinned + /// value, else the request's. What a transport reports through + /// `ToolCallLlm::effective_temperature`. + pub fn effective_temperature(&self, requested: f64) -> Option { + self.sends_temperature().then(|| self.temperature_value.unwrap_or(requested)) + } + + /// The profile the `$AREEV_RUN_LLM_*` variables describe — + /// `AREEV_RUN_LLM_TOKEN_FIELD`, `AREEV_RUN_LLM_NO_TEMPERATURE`, + /// `AREEV_RUN_LLM_EXTRA_BODY` — or `None` when none is set. The same + /// variables the CLI reads as its flags' twins, read by the bindings when + /// a host passes no `llm_profile`, so an embedding host configured out of + /// band behaves like the CLI. + pub fn from_env() -> Result> { + Self::from_lookup(&|k| std::env::var(k).ok()) + } + + /// [`from_env`](Self::from_env) over any lookup — the parser itself, so a + /// test need not touch the process environment. + pub fn from_lookup(get: &dyn Fn(&str) -> Option) -> Result> { + let get = |k: &str| get(k).map(|v| v.trim().to_string()).filter(|v| !v.is_empty()); + let mut p = RequestProfile::new(); + let mut any = false; + if let Some(f) = get("AREEV_RUN_LLM_TOKEN_FIELD") { + any = true; + p = p.token_limit(match f.as_str() { + "max_tokens" => TokenLimitField::MaxTokens, + "max_completion_tokens" => TokenLimitField::MaxCompletionTokens, + other => { + return Err(Error::LlmBackend(format!( + "AREEV_RUN_LLM_TOKEN_FIELD {other:?}: expected max_tokens or \ + max_completion_tokens" + ))) + } + }); + } + if get("AREEV_RUN_LLM_NO_TEMPERATURE").is_some_and(|v| is_truthy(&v)) { + any = true; + p = p.temperature(TemperatureMode::Omit); + } + if let Some(body) = get("AREEV_RUN_LLM_EXTRA_BODY") { + any = true; + p = p.extra_body_json(&body)?; + } + Ok(any.then_some(p)) + } + + /// Parse the bindings' `llm_profile` JSON (#394): + /// `{"temperature": "omit" | , "token_field": "max_tokens" | + /// "max_completion_tokens", "extra_body": {…}}`, every key optional. + /// + /// Unknown keys are refused by name, as `extra_body`'s reserved keys are: + /// a misspelt `temprature` that silently configured nothing is the run + /// that 400s on its first turn. + pub fn from_json(json: &str) -> Result { + let v: Value = serde_json::from_str(json) + .map_err(|e| Error::LlmBackend(format!("llm_profile is not valid JSON: {e}")))?; + let Value::Object(map) = v else { + return Err(Error::LlmBackend("llm_profile must be a JSON object".into())); + }; + let mut p = RequestProfile::new(); + for (k, v) in &map { + match k.as_str() { + "temperature" => match v { + Value::String(s) if s == "omit" => p = p.temperature(TemperatureMode::Omit), + Value::String(s) if s == "send" => p = p.temperature(TemperatureMode::Send), + Value::Number(n) => { + let t = n.as_f64().filter(|t| t.is_finite() && *t >= 0.0).ok_or_else(|| { + Error::LlmBackend(format!( + "llm_profile.temperature {n} is not a non-negative number" + )) + })?; + p = p.temperature_value(t); + } + other => { + return Err(Error::LlmBackend(format!( + "llm_profile.temperature must be \"omit\" or a number, not {other}" + ))) + } + }, + "token_field" => { + p = p.token_limit(match v.as_str() { + Some("max_tokens") => TokenLimitField::MaxTokens, + Some("max_completion_tokens") => TokenLimitField::MaxCompletionTokens, + _ => { + return Err(Error::LlmBackend(format!( + "llm_profile.token_field must be \"max_tokens\" or \ + \"max_completion_tokens\", not {v}" + ))) + } + }) + } + "extra_body" => match v { + Value::Object(m) => p = p.extra_body(m.clone())?, + _ => { + return Err(Error::LlmBackend( + "llm_profile.extra_body must be a JSON object".into(), + )) + } + }, + other => { + return Err(Error::LlmBackend(format!( + "llm_profile has no key {other:?} (known: temperature, token_field, \ + extra_body)" + ))) + } + } + } + Ok(p) + } + /// Extra top-level request fields, merged last and passed through /// untouched — `store`, `reasoning_effort`, `provider`, `thinking`, /// `output_config`, `inference_geo`, `seed`, anything a vendor adds @@ -145,6 +288,7 @@ impl RequestProfile { self.token_limit == TokenLimitField::default() && self.temperature == TemperatureMode::default() && self.extra_body.is_empty() + && self.temperature_value.is_none() } /// A stable content address of this profile, for the run manifest's LLM @@ -155,11 +299,16 @@ impl RequestProfile { } // `Map` is a BTreeMap under serde_json's default features, so the // serialization is key-sorted and the digest is stable. - let canonical = serde_json::json!({ + let mut canonical = serde_json::json!({ "token_limit": self.token_limit_key(), "temperature": if self.sends_temperature() { "send" } else { "omit" }, "extra_body": Value::Object(self.extra_body.clone()), }); + // Only when set, so every profile that existed before the pinned + // value keeps the digest a parked run was pinned to. + if let Some(t) = self.temperature_value { + canonical["temperature_value"] = serde_json::json!(t); + } let digest = areev_core::anon::hmac_sha256( b"areev-request-profile/v1", canonical.to_string().as_bytes(), @@ -225,6 +374,58 @@ mod tests { assert!(RequestProfile::new().extra_body_json("[1,2]").is_err()); } + #[test] + fn the_bindings_profile_json_parses_every_key_and_refuses_strangers() { + let p = RequestProfile::from_json( + r#"{"temperature":"omit","token_field":"max_completion_tokens","extra_body":{"store":false}}"#, + ) + .unwrap(); + assert!(!p.sends_temperature()); + assert_eq!(p.token_limit_key(), "max_completion_tokens"); + let mut body = serde_json::json!({}); + p.apply_extra(&mut body); + assert_eq!(body["store"], serde_json::json!(false)); + + let n = RequestProfile::from_json(r#"{"temperature":0.7}"#).unwrap(); + assert!(n.sends_temperature()); + assert!(!n.is_default(), "a pinned value is configuration"); + assert_ne!(n.digest(), RequestProfile::default().digest()); + + assert!(RequestProfile::new().digest().is_none()); + for bad in [ + r#"{"temprature":"omit"}"#, + r#"{"temperature":"cold"}"#, + r#"{"temperature":-1}"#, + r#"{"token_field":"tokens"}"#, + r#"{"extra_body":{"model":"x"}}"#, + r#"{"extra_body":[1]}"#, + "[]", + ] { + assert!(RequestProfile::from_json(bad).is_err(), "{bad} must be refused"); + } + } + + #[test] + fn the_env_twins_build_a_profile_and_off_means_off() { + let env = |pairs: &'static [(&'static str, &'static str)]| { + move |k: &str| pairs.iter().find(|(n, _)| *n == k).map(|(_, v)| v.to_string()) + }; + assert!(RequestProfile::from_lookup(&env(&[])).unwrap().is_none()); + for off in ["off", "no", "0", "false", "OFF"] { + let pairs: &'static [(&str, &str)] = Box::leak(Box::new([("AREEV_RUN_LLM_NO_TEMPERATURE", off)])); + assert!(RequestProfile::from_lookup(&env(pairs)).unwrap().is_none(), "{off} is off"); + } + let p = RequestProfile::from_lookup(&env(&[ + ("AREEV_RUN_LLM_NO_TEMPERATURE", "1"), + ("AREEV_RUN_LLM_TOKEN_FIELD", "max_completion_tokens"), + ])) + .unwrap() + .unwrap(); + assert!(!p.sends_temperature()); + assert_eq!(p.token_limit_key(), "max_completion_tokens"); + assert!(RequestProfile::from_lookup(&env(&[("AREEV_RUN_LLM_TOKEN_FIELD", "x")])).is_err()); + } + #[test] fn the_digest_is_stable_and_distinguishes_profiles() { let a = RequestProfile::new().extra_body_json(r#"{"a":1,"b":2}"#).unwrap(); diff --git a/crates/areev-llm/src/toolcall.rs b/crates/areev-llm/src/toolcall.rs index f589369a..9d3a1a92 100644 --- a/crates/areev-llm/src/toolcall.rs +++ b/crates/areev-llm/src/toolcall.rs @@ -163,6 +163,12 @@ pub struct Usage { } /// One tool-calling request. `max_tokens` is mandatory by design. +/// +/// `Default` exists so a caller can name the fields it sets and write +/// `..Default::default()` for the rest: the optional request fields below grew +/// in 1.13 (#391, #394) and will grow again, and a struct literal naming every +/// field breaks every host each time one is added. +#[derive(Default)] pub struct ToolCallRequest<'a> { pub system: Option, pub messages: Vec, @@ -173,6 +179,95 @@ pub struct ToolCallRequest<'a> { /// Defaults to 0.0 at every call site that wants reproducible-ish output; /// carried explicitly so the journaled request is complete. pub temperature: f64, + /// Send NO temperature on this request, whatever `temperature` says (#391). + /// + /// The GPT-5 family answers HTTP 400 "Unsupported value: 'temperature'" to + /// any value at all, so a plan node driving one says `"temperature": + /// "omit"`. The endpoint's request profile can omit it too; either one is + /// enough. + pub omit_temperature: bool, + /// Structured output the caller asks for (#391). Sent natively where the + /// wire format has a field for it (`response_format` on OpenAI-compatible + /// endpoints, `format` on Ollama) and stated in the system text where it + /// has none (Anthropic). Either way the CALLER still validates the answer: + /// a provider that ignores the field must not be able to hand back prose + /// as if it were the schema's object. + pub response_format: Option, + /// `parallel_tool_calls` (#394). `None` sends nothing and leaves the + /// provider's default; `Some(false)` asks for at most one call per turn. + /// OpenAI-compatible endpoints take the field as is; Anthropic takes the + /// inverse, `tool_choice.disable_parallel_tool_use`. + pub parallel_tool_calls: Option, +} + +/// A structured-output request (#391). +#[derive(Debug, Clone, PartialEq)] +pub enum ResponseFormat { + /// Any JSON object. + JsonObject, + /// A JSON object conforming to `schema`. `name` labels it on the wire + /// (OpenAI requires `^[a-zA-Z0-9_-]{1,64}$`; [`ResponseFormat::json_schema`] + /// sanitizes to that). + JsonSchema { name: String, schema: Value }, +} + +impl ResponseFormat { + /// A schema request whose wire name is `name` made safe for every + /// provider: characters outside `[a-zA-Z0-9_-]` become `_`, cut to 64. + pub fn json_schema(name: &str, schema: Value) -> Self { + let mut safe: String = name + .chars() + .map(|c| if c.is_ascii_alphanumeric() || c == '_' || c == '-' { c } else { '_' }) + .take(64) + .collect(); + if safe.is_empty() { + safe = "response".into(); + } + ResponseFormat::JsonSchema { name: safe, schema } + } + + /// The OpenAI-compatible `response_format` value — also the shape the + /// runtime journals as "what was asked for", so a reader of a run sees + /// the field exactly as it went out on that wire. + pub fn to_openai(&self) -> Value { + match self { + ResponseFormat::JsonObject => json!({"type": "json_object"}), + ResponseFormat::JsonSchema { name, schema } => json!({ + "type": "json_schema", + "json_schema": {"name": name, "schema": schema, "strict": true}, + }), + } + } + + /// Read back the OpenAI-compatible shape [`to_openai`](Self::to_openai) + /// writes. `None` for anything else. + pub fn from_openai(v: &Value) -> Option { + match v.get("type").and_then(Value::as_str)? { + "json_object" => Some(ResponseFormat::JsonObject), + "json_schema" => { + let js = v.get("json_schema")?; + Some(ResponseFormat::JsonSchema { + name: js.get("name").and_then(Value::as_str).unwrap_or("response").to_string(), + schema: js.get("schema")?.clone(), + }) + } + _ => None, + } + } + + /// The instruction a provider with no native field is given instead, + /// appended to the system text. + pub fn as_instruction(&self) -> String { + match self { + ResponseFormat::JsonObject => { + "Answer with a single JSON object and nothing else.".to_string() + } + ResponseFormat::JsonSchema { schema, .. } => format!( + "Answer with a single JSON object and nothing else. It must conform to this \ + JSON Schema:\n{schema}" + ), + } + } } /// A parsed tool-calling response. @@ -553,7 +648,7 @@ fn openai_tool_choice(tc: &ToolChoice) -> Value { } } -fn openai_parse(resp: &Value) -> ToolCallResult { +pub(crate) fn openai_parse(resp: &Value) -> ToolCallResult { let msg = resp .pointer("/choices/0/message") .ok_or_else(|| terminal("response has no choices[0].message"))?; @@ -625,8 +720,8 @@ fn openai_body( "stream": false, }); body[profile.token_limit_key()] = json!(req.max_tokens); - if profile.sends_temperature() { - body["temperature"] = json!(req.temperature); + if let Some(t) = profile.temperature_for(req) { + body["temperature"] = json!(t); } if !req.tools.is_empty() { body["tools"] = Value::Array(render_tools(req.tools, ProviderKind::OpenAiTools)?); @@ -635,6 +730,20 @@ fn openai_body( // Merged last, and it can never clobber an owned key — the profile // constructor already refused those (#285). profile.apply_extra(&mut body); + // The REQUEST's own fields after the profile's pass-through: a node that + // asked for `parallel_tool_calls: false` or a schema gets it even on an + // endpoint whose profile injects a different default through + // `extra_body` (which predates both fields and so may carry them). + if let Some(rf) = &req.response_format { + body["response_format"] = rf.to_openai(); + } + if let Some(p) = req.parallel_tool_calls { + // Only meaningful with tools on the request; OpenAI refuses the field + // on a request that offers none. + if !req.tools.is_empty() { + body["parallel_tool_calls"] = json!(p); + } + } Ok(body) } @@ -652,6 +761,11 @@ impl ToolCallLlm for crate::OpenAiCompat { fn request_profile_digest(&self) -> Option { self.profile.digest() } + /// What the profile actually sends (#394 review): none when it omits, + /// its pinned value when it pins one. + fn effective_temperature(&self, requested: f64) -> Option { + self.profile.effective_temperature(requested) + } fn call(&self, req: &ToolCallRequest<'_>) -> ToolCallResult { let body = openai_body(req, &self.model, &self.profile)?; let url = format!("{}/chat/completions", self.base_url.trim_end_matches('/')); @@ -757,7 +871,7 @@ fn anthropic_tool_choice(tc: &ToolChoice) -> Option { } } -fn anthropic_parse(resp: &Value) -> ToolCallResult { +pub(crate) fn anthropic_parse(resp: &Value) -> ToolCallResult { let blocks = resp .get("content") .and_then(|c| c.as_array()) @@ -871,10 +985,20 @@ fn anthropic_body( }); // The model's own contract first (#283: current Claude models 400 on a // sampling parameter), then the operator's profile (#285). - if anthropic_accepts_temperature(model) && profile.sends_temperature() { - body["temperature"] = json!(req.temperature); + if anthropic_accepts_temperature(model) { + if let Some(t) = profile.temperature_for(req) { + body["temperature"] = json!(t); + } } - if let Some(s) = &req.system { + // This wire format has no structured-output field, so a requested format + // is stated in the system text instead. The caller validates the answer + // either way — this only makes a conforming answer likelier. + let system = match (&req.system, &req.response_format) { + (Some(s), Some(rf)) => Some(format!("{s}\n\n{}", rf.as_instruction())), + (None, Some(rf)) => Some(rf.as_instruction()), + (s, None) => s.clone(), + }; + if let Some(s) = &system { // Same cache posture as the loop adapter: the stable prefix is // marked ephemeral-cacheable. body["system"] = @@ -882,7 +1006,12 @@ fn anthropic_body( } if !req.tools.is_empty() && req.tool_choice != ToolChoice::None { body["tools"] = Value::Array(render_tools(req.tools, ProviderKind::AnthropicTools)?); - if let Some(tc) = anthropic_tool_choice(&req.tool_choice) { + if let Some(mut tc) = anthropic_tool_choice(&req.tool_choice) { + // `parallel_tool_calls: false` is spelled the other way round + // here, on the tool choice. + if req.parallel_tool_calls == Some(false) { + tc["disable_parallel_tool_use"] = json!(true); + } body["tool_choice"] = tc; } } @@ -921,7 +1050,10 @@ impl ToolCallLlm for crate::Anthropic { self.model.starts_with("claude-").then_some(CLAUDE_CONTEXT_FLOOR) } fn effective_temperature(&self, requested: f64) -> Option { - anthropic_accepts_temperature(&self.model).then_some(requested) + if !anthropic_accepts_temperature(&self.model) { + return None; + } + self.profile.effective_temperature(requested) } fn request_profile_digest(&self) -> Option { self.profile.digest() @@ -974,7 +1106,7 @@ impl ToolCallLlm for crate::Anthropic { // ---- Ollama ----------------------------------------------------------------- -fn ollama_parse(resp: &Value) -> ToolCallResult { +pub(crate) fn ollama_parse(resp: &Value) -> ToolCallResult { let msg = resp .get("message") .ok_or_else(|| terminal("response has no message"))?; @@ -1039,10 +1171,21 @@ fn ollama_body(req: &ToolCallRequest<'_>, model: &str) -> ToolCallResult "stream": false, "options": {"temperature": req.temperature, "num_predict": req.max_tokens}, }); + if req.omit_temperature { + if let Some(o) = body["options"].as_object_mut() { + o.remove("temperature"); + } + } if !req.tools.is_empty() { // Ollama's native /api/chat takes OpenAI-shaped tool definitions. body["tools"] = Value::Array(render_tools(req.tools, ProviderKind::OpenAiTools)?); } + // Ollama's structured output: `format` is a JSON Schema, or "json". + match &req.response_format { + Some(ResponseFormat::JsonSchema { schema, .. }) => body["format"] = schema.clone(), + Some(ResponseFormat::JsonObject) => body["format"] = json!("json"), + None => {} + } Ok(body) } @@ -1152,6 +1295,7 @@ mod tests { tool_choice: ToolChoice::Named("receipt.prepare".into()), max_tokens: 256, temperature: 0.0, + ..Default::default() }; let openai = openai_messages(&req); @@ -1176,6 +1320,23 @@ mod tests { /// The floor is claimed for `claude-*` and nothing else: this transport /// also fronts Bedrock and Vertex model names it was never checked /// against, and a wrong-high window is the failure it exists to prevent. + #[test] + fn effective_temperature_reports_what_the_profile_sends() { + use crate::profile::{RequestProfile, TemperatureMode}; + let o = crate::OpenAiCompat::new("http://x", "k", "m"); + assert_eq!(o.effective_temperature(0.2), Some(0.2)); + let o = crate::OpenAiCompat::new("http://x", "k", "m") + .with_request_profile(RequestProfile::new().temperature(TemperatureMode::Omit)); + assert_eq!(o.effective_temperature(0.2), None); + let o = crate::OpenAiCompat::new("http://x", "k", "m") + .with_request_profile(RequestProfile::new().temperature_value(0.7)); + assert_eq!(o.effective_temperature(0.2), Some(0.7)); + let a = crate::Anthropic::new("k", "claude-sonnet-4-5") + .with_request_profile(RequestProfile::new().temperature_value(0.7)); + assert_eq!(a.effective_temperature(0.2), Some(0.7)); + assert_eq!(crate::Anthropic::new("k", "claude-opus-5").effective_temperature(0.2), None); + } + #[test] fn the_context_floor_is_claimed_only_for_claude_models() { let claude = crate::Anthropic::new("k", "claude-sonnet-5"); diff --git a/crates/areev-llm/src/toolcall_stream.rs b/crates/areev-llm/src/toolcall_stream.rs index be816fd4..94a2473a 100644 --- a/crates/areev-llm/src/toolcall_stream.rs +++ b/crates/areev-llm/src/toolcall_stream.rs @@ -75,6 +75,12 @@ pub(crate) fn openai_accumulate( let mut calls: BTreeMap = BTreeMap::new(); let mut finish: Option = None; let mut usage: Option = None; + // What the provider says it SERVED (#287, #394). Every chunk of an + // OpenAI-compatible stream repeats `model`; the first one is taken, so a + // streamed turn journals exactly the provenance a non-streamed one does — + // a run observed through `onEvent` (which turns streaming on) used to + // record less than the same run unobserved. + let mut served_model: Option = None; for line in lines { let line = line.map_err(|e| transport(format!("stream read: {e}")))?; let Some(payload) = line.strip_prefix("data:").map(str::trim) else { @@ -85,6 +91,10 @@ pub(crate) fn openai_accumulate( } let chunk: Value = serde_json::from_str(payload) .map_err(|e| terminal(format!("stream chunk does not parse: {e}")))?; + if served_model.is_none() { + served_model = + chunk.get("model").and_then(Value::as_str).filter(|m| !m.is_empty()).map(str::to_string); + } if let Some(delta) = chunk.pointer("/choices/0/delta") { if let Some(t) = delta.get("content").and_then(|c| c.as_str()) { if !t.is_empty() { @@ -151,7 +161,10 @@ pub(crate) fn openai_accumulate( tool_calls, stop_reason, usage, - )) + ) + // No OpenAI-compatible wire format names a serving region, streamed or + // not; `None` matches `openai_parse`. + .with_served(served_model, None)) } // ---- Anthropic (SSE, typed data events) ------------------------------------ @@ -183,6 +196,9 @@ pub(crate) fn anthropic_accumulate( let mut output_tokens = 0u64; let mut cache_read: Option = None; let mut saw_usage = false; + // `message_start` carries the message object, `model` included — the same + // field `anthropic_parse` reads off a whole response (#394). + let mut served_model: Option = None; for line in lines { let line = line.map_err(|e| transport(format!("stream read: {e}")))?; let Some(payload) = line.strip_prefix("data:").map(str::trim) else { @@ -192,6 +208,10 @@ pub(crate) fn anthropic_accumulate( .map_err(|e| terminal(format!("stream event does not parse: {e}")))?; match ev.get("type").and_then(|t| t.as_str()) { Some("message_start") => { + served_model = ev + .pointer("/message/model") + .and_then(Value::as_str) + .map(str::to_string); if let Some(u) = ev.pointer("/message/usage") { input_tokens = u.get("input_tokens").and_then(|v| v.as_u64()).unwrap_or(0); cache_read = @@ -351,7 +371,8 @@ pub(crate) fn anthropic_accumulate( ) // Only when the turn actually held opaque content, so a stream of plain // text and tool calls journals exactly what it did before #284. - .with_provider_content(has_opaque.then_some(Value::Array(content)))) + .with_provider_content(has_opaque.then_some(Value::Array(content))) + .with_served(served_model, None)) } // ---- Ollama (NDJSON) ------------------------------------------------------- @@ -364,6 +385,8 @@ pub(crate) fn ollama_accumulate( let mut tool_calls: Vec = Vec::new(); let mut done_reason: Option = None; let mut usage: Option = None; + // Every NDJSON line names the model, as the whole response does (#394). + let mut served_model: Option = None; for line in lines { let line = line.map_err(|e| transport(format!("stream read: {e}")))?; if line.trim().is_empty() { @@ -371,6 +394,9 @@ pub(crate) fn ollama_accumulate( } let chunk: Value = serde_json::from_str(&line) .map_err(|e| terminal(format!("stream line does not parse: {e}")))?; + if served_model.is_none() { + served_model = chunk.get("model").and_then(Value::as_str).map(str::to_string); + } if let Some(t) = chunk.pointer("/message/content").and_then(|c| c.as_str()) { if !t.is_empty() { on_token(t); @@ -429,7 +455,8 @@ pub(crate) fn ollama_accumulate( tool_calls, stop_reason, usage, - )) + ) + .with_served(served_model, None)) } #[cfg(test)] @@ -553,4 +580,58 @@ data: {"type":"message_stop"} assert_eq!(resp.usage.input_tokens, 21); assert_eq!(resp.usage.output_tokens, 5); } + /// #394: a streamed turn records the provenance the same turn records + /// unstreamed — `served_model` from the response's `model`, and the same + /// (absent) `served_region` — on all three wire formats. A run observed + /// through `onEvent` streams, so before this it journaled less than the + /// identical run with no observer. + #[test] + fn a_streamed_turn_records_the_provenance_a_whole_response_does() { + let whole = serde_json::json!({ + "model": "gpt-5-mini-2026-08-07", + "choices": [{"message": {"content": "hi"}, "finish_reason": "stop"}], + "usage": {"prompt_tokens": 3, "completion_tokens": 1}, + }); + let streamed = r#"data: {"model":"gpt-5-mini-2026-08-07","choices":[{"delta":{"content":"hi"}}]} +data: {"model":"gpt-5-mini-2026-08-07","choices":[{"delta":{},"finish_reason":"stop"}]} +data: {"model":"gpt-5-mini-2026-08-07","choices":[],"usage":{"prompt_tokens":3,"completion_tokens":1}} +data: [DONE] +"#; + let a = crate::toolcall::openai_parse(&whole).unwrap(); + let b = openai_accumulate(lines(streamed), &mut |_| {}).unwrap(); + assert_eq!(a.served_model.as_deref(), Some("gpt-5-mini-2026-08-07")); + assert_eq!((a.served_model, a.served_region), (b.served_model, b.served_region)); + + let whole = serde_json::json!({ + "model": "claude-sonnet-5-20260901", + "content": [{"type": "text", "text": "hi"}], + "stop_reason": "end_turn", + "usage": {"input_tokens": 3, "output_tokens": 1}, + }); + let streamed = r#"data: {"type":"message_start","message":{"model":"claude-sonnet-5-20260901","usage":{"input_tokens":3}}} +data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}} +data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"hi"}} +data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":1}} +data: {"type":"message_stop"} +"#; + let a = crate::toolcall::anthropic_parse(&whole).unwrap(); + let b = anthropic_accumulate(lines(streamed), &mut |_| {}).unwrap(); + assert_eq!(a.served_model.as_deref(), Some("claude-sonnet-5-20260901")); + assert_eq!((a.served_model, a.served_region), (b.served_model, b.served_region)); + + let whole = serde_json::json!({ + "model": "llama3.1:8b", + "message": {"content": "hi"}, + "done_reason": "stop", + "prompt_eval_count": 3, + "eval_count": 1, + }); + let streamed = r#"{"model":"llama3.1:8b","message":{"content":"hi"},"done":false} +{"model":"llama3.1:8b","message":{"content":""},"done":true,"done_reason":"stop","prompt_eval_count":3,"eval_count":1} +"#; + let a = crate::toolcall::ollama_parse(&whole).unwrap(); + let b = ollama_accumulate(lines(streamed), &mut |_| {}).unwrap(); + assert_eq!(a.served_model.as_deref(), Some("llama3.1:8b")); + assert_eq!((a.served_model, a.served_region), (b.served_model, b.served_region)); + } } diff --git a/crates/areev-llm/tests/toolcall_fixture_tests.rs b/crates/areev-llm/tests/toolcall_fixture_tests.rs index e886bce2..6cfb9c16 100644 --- a/crates/areev-llm/tests/toolcall_fixture_tests.rs +++ b/crates/areev-llm/tests/toolcall_fixture_tests.rs @@ -145,6 +145,7 @@ fn req<'a>(tools: &'a [Tool]) -> ToolCallRequest<'a> { tool_choice: ToolChoice::Auto, max_tokens: 256, temperature: 0.0, + ..Default::default() } } @@ -326,6 +327,7 @@ fn anthropic_tool_use_blocks_round_trip_with_history() { tool_choice: ToolChoice::Required, max_tokens: 512, temperature: 0.0, + ..Default::default() }; let resp = llm.call(&request).unwrap(); @@ -581,6 +583,7 @@ fn a_replayed_assistant_turn_sends_the_provider_content_byte_identically() { tool_choice: ToolChoice::Auto, max_tokens: 256, temperature: 0.0, + ..Default::default() }; llm.call(&request).unwrap(); let sent = rx.recv().unwrap(); @@ -801,3 +804,132 @@ fn a_default_credential_still_uses_the_adapters_own_header() { .unwrap() .is_none()); } + +// ── #391/#394: per-request structured output, temperature, parallel calls ── + +fn openai_ok() -> Value { + json!({ + "choices": [{"message": {"content": "{}"}, "finish_reason": "stop"}], + "usage": {"prompt_tokens": 1, "completion_tokens": 1} + }) +} + +#[test] +fn openai_sends_the_requested_schema_parallel_flag_and_no_temperature() { + use areev_llm::toolcall::ResponseFormat; + let (base, rx) = one_shot_server(200, openai_ok()); + let llm = OpenAiCompat::new(&base, "k", "gpt-5-mini"); + let tools = [weather_tool()]; + let schema = json!({"type": "object", "properties": {"a": {"type": "string"}}, "required": ["a"]}); + let mut r = req(&tools); + r.response_format = Some(ResponseFormat::json_schema("reply.node", schema.clone())); + r.parallel_tool_calls = Some(false); + r.omit_temperature = true; + llm.call(&r).unwrap(); + let sent = rx.recv().unwrap(); + assert_eq!(sent["response_format"]["type"], "json_schema"); + assert_eq!(sent["response_format"]["json_schema"]["name"], "reply_node", "sanitized"); + assert_eq!(sent["response_format"]["json_schema"]["strict"], true); + assert_eq!(sent["response_format"]["json_schema"]["schema"], schema); + assert_eq!(sent["parallel_tool_calls"], false); + assert!(sent.get("temperature").is_none(), "omit sends no key at all: {sent}"); + + // json:true is json_object; no tools means no parallel_tool_calls key + // (OpenAI refuses the field on a request that offers none). + let (base, rx) = one_shot_server(200, openai_ok()); + let llm = OpenAiCompat::new(&base, "k", "m"); + let mut r = req(&[]); + r.response_format = Some(ResponseFormat::JsonObject); + r.parallel_tool_calls = Some(true); + r.temperature = 0.3; + llm.call(&r).unwrap(); + let sent = rx.recv().unwrap(); + assert_eq!(sent["response_format"], json!({"type": "json_object"})); + assert!(sent.get("parallel_tool_calls").is_none()); + assert_eq!(sent["temperature"], 0.3); +} + +#[test] +fn a_request_field_wins_over_a_profile_extra_body_carrying_the_same_key() { + use areev_llm::profile::RequestProfile; + let (base, rx) = one_shot_server(200, openai_ok()); + let profile = RequestProfile::new() + .extra_body_json(r#"{"parallel_tool_calls":true}"#) + .unwrap(); + let llm = OpenAiCompat::new(&base, "k", "m").with_request_profile(profile); + let tools = [weather_tool()]; + let mut r = req(&tools); + r.parallel_tool_calls = Some(false); + llm.call(&r).unwrap(); + assert_eq!(rx.recv().unwrap()["parallel_tool_calls"], false); +} + +#[test] +fn a_profile_temperature_value_is_sent_and_either_omission_wins() { + use areev_llm::profile::{RequestProfile, TemperatureMode}; + let (base, rx) = one_shot_server(200, openai_ok()); + let llm = OpenAiCompat::new(&base, "k", "m") + .with_request_profile(RequestProfile::new().temperature_value(0.7)); + let tools = [weather_tool()]; + llm.call(&req(&tools)).unwrap(); + assert_eq!(rx.recv().unwrap()["temperature"], 0.7); + + let (base, rx) = one_shot_server(200, openai_ok()); + let llm = OpenAiCompat::new(&base, "k", "m") + .with_request_profile(RequestProfile::new().temperature_value(0.7)); + let mut r = req(&tools); + r.omit_temperature = true; + llm.call(&r).unwrap(); + assert!(rx.recv().unwrap().get("temperature").is_none()); + + let (base, rx) = one_shot_server(200, openai_ok()); + let llm = OpenAiCompat::new(&base, "k", "m") + .with_request_profile(RequestProfile::new().temperature(TemperatureMode::Omit)); + let mut r = req(&tools); + r.temperature = 0.4; + llm.call(&r).unwrap(); + assert!(rx.recv().unwrap().get("temperature").is_none()); +} + +#[test] +fn anthropic_states_the_schema_in_the_system_text_and_disables_parallel_use() { + use areev_llm::toolcall::ResponseFormat; + let (base, rx) = one_shot_server(200, anthropic_ok_body()); + let llm = Anthropic::new("k", "claude-sonnet-4-5").with_base_url(&base); + let tools = [weather_tool()]; + let mut r = req(&tools); + r.system = Some("You are the AP assistant.".into()); + r.response_format = Some(ResponseFormat::json_schema( + "reply", + json!({"type": "object", "properties": {"answer": {"type": "string"}}}), + )); + r.parallel_tool_calls = Some(false); + r.omit_temperature = true; + llm.call(&r).unwrap(); + let sent = rx.recv().unwrap(); + let system = sent["system"][0]["text"].as_str().unwrap(); + assert!(system.starts_with("You are the AP assistant."), "{system}"); + assert!(system.contains("\"answer\""), "the schema rides the system text: {system}"); + assert!(sent.get("response_format").is_none(), "no such field on this wire"); + assert_eq!(sent["tool_choice"]["disable_parallel_tool_use"], true); + assert!(sent.get("temperature").is_none(), "a legacy model still omits on request"); +} + +#[test] +fn ollama_takes_the_schema_as_format_and_omits_the_temperature_option() { + use areev_llm::toolcall::ResponseFormat; + let (base, rx) = one_shot_server( + 200, + json!({"message": {"content": "{}"}, "done_reason": "stop", + "prompt_eval_count": 1, "eval_count": 1}), + ); + let llm = Ollama::new(&base, "local-model"); + let schema = json!({"type": "object", "properties": {}}); + let mut r = req(&[]); + r.response_format = Some(ResponseFormat::json_schema("n", schema.clone())); + r.omit_temperature = true; + llm.call(&r).unwrap(); + let sent = rx.recv().unwrap(); + assert_eq!(sent["format"], schema); + assert!(sent["options"].get("temperature").is_none()); +} diff --git a/crates/areev-loop-adapter/Cargo.toml b/crates/areev-loop-adapter/Cargo.toml index 0b8822be..3cf9a41d 100644 --- a/crates/areev-loop-adapter/Cargo.toml +++ b/crates/areev-loop-adapter/Cargo.toml @@ -13,14 +13,14 @@ keywords = ["ai-agents", "memory", "loop", "self-improvement", "areev"] categories = ["algorithms"] [dependencies] -areev-loop = { path = "../areev-loop", version = "1.12.0" } -areev-core = { path = "../areev-core", version = "1.12.0" } -areev-store = { path = "../areev-store", version = "1.12.0" } -areev-cal = { path = "../areev-cal", version = "1.12.0" } -areev-run-core = { path = "../areev-run-core", version = "1.12.0" } +areev-loop = { path = "../areev-loop", version = "1.13.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } +areev-store = { path = "../areev-store", version = "1.13.0" } +areev-cal = { path = "../areev-cal", version = "1.13.0" } +areev-run-core = { path = "../areev-run-core", version = "1.13.0" } # The plan-change rehearsal a `plan_revision` proposal carries: the runtime's # shadow over the live plan's journaled runs (`areev run shadow --plan-file`). -areev-run = { path = "../areev-run", version = "1.12.0" } +areev-run = { path = "../areev-run", version = "1.13.0" } serde = { version = "1", features = ["derive"] } serde_json = "1" diff --git a/crates/areev-mcp/Cargo.toml b/crates/areev-mcp/Cargo.toml index 19875c2a..688932b2 100644 --- a/crates/areev-mcp/Cargo.toml +++ b/crates/areev-mcp/Cargo.toml @@ -13,18 +13,18 @@ keywords = ["mcp", "memory", "ai-agents", "llm", "protocol"] categories = ["api-bindings"] [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } -areev-store = { path = "../areev-store", version = "1.12.0" } -areev-cal = { path = "../areev-cal", version = "1.12.0" } -areev-loop = { path = "../areev-loop", version = "1.12.0" } -areev-loop-adapter = { path = "../areev-loop-adapter", version = "1.12.0" } -areev-run = { path = "../areev-run", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } +areev-store = { path = "../areev-store", version = "1.13.0" } +areev-cal = { path = "../areev-cal", version = "1.13.0" } +areev-loop = { path = "../areev-loop", version = "1.13.0" } +areev-loop-adapter = { path = "../areev-loop-adapter", version = "1.13.0" } +areev-run = { path = "../areev-run", version = "1.13.0" } # The decision-backend chain ($AREEV_DECIDE*, docs/decision-model-proposal.md). # Already in this crate's graph via areev-run; `default-features = false` for # the same reason areev-run gives (#289): this crate only resolves the chain, # and which LLM providers an `llm:` entry can reach is the HOST's feature # choice (the `areev` binary keeps the defaults). -areev-llm = { path = "../areev-llm", version = "1.12.0", default-features = false } +areev-llm = { path = "../areev-llm", version = "1.13.0", default-features = false } serde_json = "1" [dev-dependencies] diff --git a/crates/areev-py/Cargo.toml b/crates/areev-py/Cargo.toml index 4719d669..15db0f9c 100644 --- a/crates/areev-py/Cargo.toml +++ b/crates/areev-py/Cargo.toml @@ -38,9 +38,9 @@ crate-type = ["cdylib"] doc = false [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } -areev-store = { path = "../areev-store", version = "1.12.0" } -areev-cal = { path = "../areev-cal", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } +areev-store = { path = "../areev-store", version = "1.13.0" } +areev-cal = { path = "../areev-cal", version = "1.13.0" } areev-loop = { path = "../areev-loop" } areev-loop-adapter = { path = "../areev-loop-adapter" } areev-llm = { path = "../areev-llm" } diff --git a/crates/areev-py/pyproject.toml b/crates/areev-py/pyproject.toml index e73762d1..b15c52d4 100644 --- a/crates/areev-py/pyproject.toml +++ b/crates/areev-py/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "maturin" [project] name = "areev" -version = "1.12.2" +version = "1.13.0" description = "Python bindings for Areev, the embedded memory engine for AI agents." requires-python = ">=3.9" license = { text = "MIT OR Apache-2.0" } diff --git a/crates/areev-py/src/lib.rs b/crates/areev-py/src/lib.rs index 86ce40af..61259152 100644 --- a/crates/areev-py/src/lib.rs +++ b/crates/areev-py/src/lib.rs @@ -326,12 +326,34 @@ fn resolve_toolcall_llm( model: Option, base_url: Option, key_env: Option, + llm_profile: Option, ) -> PyResult>> { + // The request profile (#394) — `{"temperature": "omit" | n, "token_field": + // …, "extra_body": {…}}` — the CLI's `--llm-*` flags as one JSON string, + // refused by name when malformed (the Node binding's twin). + let explicit = llm_profile.as_deref().map(str::trim).filter(|s| !s.is_empty()).is_some(); + let profile = match llm_profile.as_deref().map(str::trim).filter(|s| !s.is_empty()) { + Some(raw) => Some(areev_llm::profile::RequestProfile::from_json(raw).map_err(err)?), + // No argument: the `$AREEV_RUN_LLM_*` twins the CLI reads, so a host + // configured out of band behaves like `areev run` (#394 review) — + // read only when there is a model to shape, so a stale, malformed + // variable cannot fail a run that calls no model at all. + None if model.is_some() => areev_llm::profile::RequestProfile::from_env().map_err(err)?, + None => None, + }; match model { Some(spec) => Ok(Some( - areev_llm::resolve_toolcall(&spec, base_url.as_deref(), key_env.as_deref()) - .map_err(err)?, + areev_llm::resolve_toolcall_with_profile( + &spec, + base_url.as_deref(), + key_env.as_deref(), + profile, + ) + .map_err(err)?, )), + None if explicit => { + Err(err("llm_profile shapes a model's requests — pass `model` with it")) + } None => Ok(None), } } @@ -2279,6 +2301,16 @@ impl Areev { /// Note `RunFinished` is emitted at a TERMINAL outcome: a run that parks /// on a human gate ends this leg at `AskRaised`, and the `run_resume` leg /// carries `RunResumed` … `RunFinished`. + /// + /// `llm_profile` (#394) shapes every request the model transport sends, + /// as a JSON string: `{"temperature": "omit" | , "token_field": + /// "max_tokens" | "max_completion_tokens", "extra_body": {…}}`, each key + /// optional — the CLI's `--llm-no-temperature` / `--llm-token-field` / + /// `--llm-extra-body`. `'{"temperature": "omit"}'` is what runs the GPT-5 + /// family, which answers 400 to any temperature. An unknown key is + /// refused. The run pins the profile's digest, so `run_resume` must be + /// given the same one. A plan's per-node `llm` block (`docs/run.md`) + /// configures abstract nodes without it. #[pyo3(signature = (workflow, run_id, input_json = None, tool_cmd = None, max_tokens = None, max_usd_micros = None, max_wall_ms = None, ask_ttl_sec = None, model = None, base_url = None, key_env = None, @@ -2287,7 +2319,8 @@ impl Areev { on_event = None, credentials = None, allow_hosts = None, tool_egress = None, credential_ttl_secs = None, resolver_env = None, max_effects_per_attempt = None, - llm_tool_result_chars = None, llm_context_tokens = None))] + llm_tool_result_chars = None, llm_context_tokens = None, + llm_profile = None))] #[allow(clippy::too_many_arguments)] fn run_start( &self, @@ -2320,6 +2353,7 @@ impl Areev { max_effects_per_attempt: Option, llm_tool_result_chars: Option, llm_context_tokens: Option, + llm_profile: Option, ) -> PyResult { let input: serde_json::Value = match input_json { Some(raw) => serde_json::from_str(&raw).map_err(|e| err(format!("input_json: {e}")))?, @@ -2328,7 +2362,7 @@ impl Areev { let h = areev_core::error::Hash::from_hex(&workflow).map_err(err)?; // Resolved before the run starts, so a bad model spec or a missing key // fails without journaling a run that cannot advance. - let llm = resolve_toolcall_llm(model, base_url, key_env)?; + let llm = resolve_toolcall_llm(model, base_url, key_env, llm_profile)?; let egress = self.egress_handle(&EgressPin { credentials, allow_hosts, tool_egress, credential_ttl_secs, resolver_env, })?; @@ -2371,7 +2405,8 @@ impl Areev { key_env = None, llm_max_tokens = None, allow_executor = None, executor_cache = None, sandbox_cmd = None, executor_timeout_secs = None, tool_env = None, on_event = None, credentials = None, allow_hosts = None, - tool_egress = None, credential_ttl_secs = None, resolver_env = None))] + tool_egress = None, credential_ttl_secs = None, resolver_env = None, + llm_profile = None))] #[allow(clippy::too_many_arguments)] // a flat FFI surface; each knob is a distinct scalar fn run_resume( &self, @@ -2393,8 +2428,11 @@ impl Areev { tool_egress: Option, credential_ttl_secs: Option, resolver_env: Option, + // The profile `run_start` took (#394): the run pins its digest, so a + // resume passes the same one or is refused RUN-E025 before the lease. + llm_profile: Option, ) -> PyResult { - let llm = resolve_toolcall_llm(model, base_url, key_env)?; + let llm = resolve_toolcall_llm(model, base_url, key_env, llm_profile)?; let egress = self.egress_handle(&EgressPin { credentials, allow_hosts, tool_egress, credential_ttl_secs, resolver_env, })?; @@ -3577,7 +3615,8 @@ impl Areev { tool_env = None, credentials = None, allow_hosts = None, tool_egress = None, credential_ttl_secs = None, resolver_env = None, max_effects_per_attempt = None, - llm_tool_result_chars = None, llm_context_tokens = None))] + llm_tool_result_chars = None, llm_context_tokens = None, + llm_profile = None))] #[allow(clippy::too_many_arguments)] fn trigger_run( &self, @@ -3611,9 +3650,11 @@ impl Areev { max_effects_per_attempt: Option, llm_tool_result_chars: Option, llm_context_tokens: Option, + // Appended (#394 review): the request profile `run_start` takes. + llm_profile: Option, ) -> PyResult { let ev = self.evaluator( - connector_cmd, tool_cmd, credentials_json, model, base_url, key_env, + connector_cmd, tool_cmd, credentials_json, model, base_url, key_env, llm_profile, ExecutorPin { allow_executor, executor_cache, sandbox_cmd, executor_timeout_secs, tool_env }, EgressPin { credentials, allow_hosts, tool_egress, credential_ttl_secs, resolver_env }, run_options(max_tokens, max_usd_micros, max_wall_ms, ask_ttl_sec, @@ -3651,7 +3692,8 @@ impl Areev { tool_env = None, credentials = None, allow_hosts = None, tool_egress = None, credential_ttl_secs = None, resolver_env = None, max_effects_per_attempt = None, - llm_tool_result_chars = None, llm_context_tokens = None))] + llm_tool_result_chars = None, llm_context_tokens = None, + llm_profile = None))] #[allow(clippy::too_many_arguments)] fn trigger_deliver( &self, @@ -3682,11 +3724,13 @@ impl Areev { max_effects_per_attempt: Option, llm_tool_result_chars: Option, llm_context_tokens: Option, + // Appended (#394 review): the request profile `run_start` takes. + llm_profile: Option, ) -> PyResult { let payload: serde_json::Value = serde_json::from_str(&payload_json) .map_err(|e| err(format!("payload_json is not JSON: {e}")))?; let ev = self.evaluator( - connector_cmd, tool_cmd, credentials_json, model, base_url, key_env, + connector_cmd, tool_cmd, credentials_json, model, base_url, key_env, llm_profile, ExecutorPin { allow_executor, executor_cache, sandbox_cmd, executor_timeout_secs, tool_env }, EgressPin { credentials, allow_hosts, tool_egress, credential_ttl_secs, resolver_env }, run_options(max_tokens, max_usd_micros, max_wall_ms, ask_ttl_sec, @@ -3819,6 +3863,7 @@ impl Areev { model: Option, base_url: Option, key_env: Option, + llm_profile: Option, pin: ExecutorPin, egress: EgressPin, opts: areev_run::RunOptions, @@ -3842,7 +3887,7 @@ impl Areev { std::sync::Arc::new(ce) as std::sync::Arc }); - let llm = resolve_toolcall_llm(model, base_url, key_env)?; + let llm = resolve_toolcall_llm(model, base_url, key_env, llm_profile)?; // A firing gets the runner `run_start` builds, pin included (#90). // Gating on `tool_cmd` alone was the same reduction one level down: a // plan whose nodes are all pinned code, or all abstract, needs no diff --git a/crates/areev-py/tests/test_run.py b/crates/areev-py/tests/test_run.py index 87a4f95e..1bcbf5c8 100644 --- a/crates/areev-py/tests/test_run.py +++ b/crates/areev-py/tests/test_run.py @@ -208,3 +208,43 @@ def read(axis, at_from): assert pinned["world_at_loss"]["executor"] == "memory" assert pinned["world_at_loss"]["read"]["axis"] == "world" assert json.loads(db.run_verify("claim-8801"))["verified"] is True + + +def test_run_start_and_resume_take_an_llm_profile_and_refuse_a_malformed_one(db): + # #394: the request profile, as a JSON string — the CLI's --llm-* flags. + wf = _plan(db) + with pytest.raises(ValueError, match="temprature"): + db.run_start(wf, "py-p1", model="ollama:llama3.1", llm_profile='{"temprature": "omit"}') + with pytest.raises(ValueError, match="llm_profile"): + db.run_start(wf, "py-p2", model="ollama:llama3.1", llm_profile="not json") + # A profile the ollama transport cannot honour is refused, never ignored. + with pytest.raises(ValueError, match="no request profile"): + db.run_start(wf, "py-p3", model="ollama:llama3.1", llm_profile='{"temperature": "omit"}') + with pytest.raises(ValueError, match="pass `model`"): + db.run_start(wf, "py-p4", llm_profile='{"temperature": "omit"}') + with pytest.raises(ValueError, match="token_field"): + db.run_resume("py-p1", model="ollama:llama3.1", llm_profile='{"token_field": "tokens"}') + assert json.loads(db.run_list(10)) == [], "no run was journaled" + + +def test_trigger_run_takes_llm_profile_and_the_env_twins_stand_in(db, monkeypatch): + # #394 review: the trigger path takes the profile; with none, the bindings + # read the CLI's $AREEV_RUN_LLM_* twins, and `off` means off. + with pytest.raises(ValueError, match="temprature"): + db.trigger_run(model="ollama:llama3.1", llm_profile='{"temprature": "omit"}') + wf = _plan(db) + monkeypatch.setenv("AREEV_RUN_LLM_NO_TEMPERATURE", "1") + with pytest.raises(ValueError, match="no request profile"): + db.run_start(wf, "py-env", model="ollama:llama3.1") + monkeypatch.setenv("AREEV_RUN_LLM_NO_TEMPERATURE", "off") + db.trigger_run(model="ollama:llama3.1") # no profile, nothing to refuse + + +def test_a_malformed_env_twin_cannot_fail_a_run_with_no_model(db, monkeypatch): + # Delta review 4: the twins are read only when a model is configured. + monkeypatch.setenv("AREEV_RUN_LLM_TOKEN_FIELD", "tokens") + wf = _plan(db) + session = json.loads(db.run_start(wf, "py-nomodel", tool_cmd='printf \'{"greeting": "hi"}\'')) + assert "parked" in session or "finished" in session + with pytest.raises(ValueError, match="AREEV_RUN_LLM_TOKEN_FIELD"): + db.run_start(wf, "py-model", model="ollama:llama3.1") diff --git a/crates/areev-run-core/CLAUDE.md b/crates/areev-run-core/CLAUDE.md index 0daa9b2c..489cf3b7 100644 --- a/crates/areev-run-core/CLAUDE.md +++ b/crates/areev-run-core/CLAUDE.md @@ -187,6 +187,38 @@ replays identically, which is exactly what the epoch is about. in the manifest); merge order is static results by node index, then Send results by task path. +## The per-node `llm` block (#391–#393) — pinned + +- `NodeExecutor::Abstract { tools, llm }`. `llm` is the frozen + `LlmNodeConfig`, and `None` must build byte-identical commands and state + (pinned by `an_empty_block_and_no_block_build_identical_commands`), which is + why the epoch did not move. +- `messages[0]` comes from `first_message`: `instruction`, or + `instruction_from` (a pointer into the node's input; a miss is + `fail_abstract_coded("SchemaValidationFailed: …")` before any effect), then + `select_state`. `$inbox` always rides along. +- The request fields ride every turn as `input.llm` (`request_block`). A fold + turn takes system, temperature and max_tokens but not response_format or + parallel_tool_calls. `reserve_for` uses the node's `max_tokens` for the + per-dispatch reservation. +- A `response` declaration gets `check_response` on the final text. One + re-prompt (`AbstractFlow.response_strikes`, with the model's answer kept as + an assistant entry), then the node fails `SchemaValidationFailed`. +- `tool_results_into`: `AbstractFlow.tool_log` gets an entry per settled flow + tool, inserted by `effect_seq` (never arrival order), with the WHOLE result + and the dispatched input. It is moved into the result at completion. +- `OfferedTool.locked` / `.dynamic`: `validated_dispatch_input` checks a + dynamic tool's own schema, merges (`dispatch_input`: wrap under + `args_into`, then the lock wins), checks the merged input against the + frozen `definition_schema`, then runs `validate_args` under + `dispatch_name()`. Only the model's arguments stay in state and transcript. + `dispatch_flow_tool` re-derives the merged input. +- `narrow` (#396): `LlmNodeConfig::narrow_top_k` (`None` = never ask) gates + `emit_offer_decide` and sets the K `narrow_offer` keeps. `true` normalizes + to absent. A dynamic tool's question text is its own description's first + line, read off the OFFER (two nodes may share a dynamic name), not + `DecideEnv.tool_descriptions`. + ## Decisions (C1–C3, `docs/decision-model-proposal.md`) — pinned A decision is an EFFECT, never a call: `NodeExecutor::Decide { tool_hash, diff --git a/crates/areev-run-core/Cargo.toml b/crates/areev-run-core/Cargo.toml index df08e630..d3b924eb 100644 --- a/crates/areev-run-core/Cargo.toml +++ b/crates/areev-run-core/Cargo.toml @@ -26,7 +26,7 @@ keywords = ["ai-agents", "workflow", "deterministic", "runtime", "areev"] categories = ["algorithms"] [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } serde = { version = "1", features = ["derive"] } serde_json = "1" sha2 = "0.11" diff --git a/crates/areev-run-core/src/lib.rs b/crates/areev-run-core/src/lib.rs index d1213cfa..58924e12 100644 --- a/crates/areev-run-core/src/lib.rs +++ b/crates/areev-run-core/src/lib.rs @@ -47,6 +47,7 @@ pub use step::{ }; pub use types::{ Ask, Budgets, Command, DecisionRecord, EdgeOutcome, EffectKind, EffectOutcome, EventIn, - FailCause, FoldRecord, JournalKey, NodeExecutor, OfferedTool, RunOutcome, DECIDE_TOOL, + DynamicDispatch, DynamicTool, FailCause, FoldRecord, JournalKey, LlmNodeConfig, NodeExecutor, + OfferedTool, RunOutcome, DECIDE_TOOL, DECIDE_URI, INBOX, PARKED_ASKS, }; diff --git a/crates/areev-run-core/src/state.rs b/crates/areev-run-core/src/state.rs index 12cfcdf4..33ba6913 100644 --- a/crates/areev-run-core/src/state.rs +++ b/crates/areev-run-core/src/state.rs @@ -255,6 +255,20 @@ pub struct AbstractFlow { /// filters the offered Definitions by. `None` = the full pinned set. #[serde(default, skip_serializing_if = "Option::is_none")] pub offer: Option, + /// Corrective re-prompts issued for a final answer that failed the node's + /// `response` schema (#391): ONE, then the node fails + /// `SchemaValidationFailed`. Zero — and absent — on every flow whose node + /// asks for no structured output. + #[serde(default, skip_serializing_if = "is_zero_u32")] + pub response_strikes: u32, + /// Every settled tool call of this attempt, in `effect_seq` order (never + /// arrival order — results of one round land in any order), for a + /// node whose `llm` block names `tool_results_into` (#391): + /// `{name, tool_call_id, model_call_id, input, result, is_error, + /// effect_seq}`. Built from the journaled outcomes as they resolve, so + /// `verify` rebuilds it byte for byte. Empty — and absent — otherwise. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub tool_log: Vec, } fn is_zero(n: &u64) -> bool { @@ -421,6 +435,8 @@ mod tests { deciding: None, decide_fold_tried: false, offer: None, + response_strikes: 0, + tool_log: Vec::new(), } } diff --git a/crates/areev-run-core/src/step.rs b/crates/areev-run-core/src/step.rs index fb19dba0..2e9881fc 100644 --- a/crates/areev-run-core/src/step.rs +++ b/crates/areev-run-core/src/step.rs @@ -51,7 +51,8 @@ use crate::state::{ }; use crate::types::{ Ask, Budgets, Command, DecisionRecord, EdgeOutcome, EffectKind, EffectOutcome, EventIn, - FailCause, FoldRecord, JournalKey, NodeExecutor, OfferedTool, RunOutcome, DECIDE_TOOL, + FailCause, FoldRecord, JournalKey, LlmNodeConfig, NodeExecutor, OfferedTool, RunOutcome, + DECIDE_TOOL, }; use areev_core::format::tool_schema::normalize_tool_name; use serde_json::Value; @@ -455,15 +456,22 @@ fn flow_attempt(st: &SchedulerState, i: usize, path: &str) -> u32 { /// Open a fresh LLM loop for a node or one Send task against an abstract /// node, and emit its first turn. fn start_flow(env: &StepEnv<'_>, st: &mut SchedulerState, i: usize, path: &str, input: Value, out: &mut Vec) { + // The task, as the node's `llm` block shapes it (#391). A pointer that + // lands on nothing fails the node here, before any effect — the same + // state would fail the same way on every retry, so it is not one. + let content = match first_message(env, i, &input) { + Ok(c) => c, + Err(why) => { + fail_abstract_coded(env, st, i, path, &format!("SchemaValidationFailed: {why}")); + return; + } + }; st.abstract_flows.insert( flow_key(i, path), crate::state::AbstractFlow { messages: vec![serde_json::json!({ "role": "user", - "content": { - "instruction": env.plan.nodes[i], - "state": input, - } + "content": content, })], next_effect_seq: 0, pending_tools: BTreeMap::new(), @@ -477,6 +485,8 @@ fn start_flow(env: &StepEnv<'_>, st: &mut SchedulerState, i: usize, path: &str, deciding: None, decide_fold_tried: false, offer: None, + response_strikes: 0, + tool_log: Vec::new(), }, ); // C1: with a calibrated backend and more than OFFER_TOP_K pinned tools, @@ -488,6 +498,288 @@ fn start_flow(env: &StepEnv<'_>, st: &mut SchedulerState, i: usize, path: &str, dispatch_llm_turn(env, st, i, path, out); } +// ---- the per-node `llm` block (#391, #392) ----------------------------------- +// +// Everything here is a pure function of the frozen block and the node's input, +// and every choice it makes that reaches the provider RIDES THE JOURNAL — the +// first message in `messages[0]`, the request fields in each turn's `llm` — +// so the driver builds the request from the journaled input alone and verify +// needs nothing else. A node with no block takes none of these paths and +// builds byte-identical commands and state. + +/// The node's frozen `llm` block, if it has one. +fn llm_cfg<'e>(env: &'e StepEnv<'_>, i: usize) -> Option<&'e LlmNodeConfig> { + match &env.executors[i] { + NodeExecutor::Abstract { llm, .. } => llm.as_deref(), + _ => None, + } +} + +/// The per-dispatch token reservation for node `i`'s turns: its own +/// `max_tokens` when the block sets one (it is that turn's output ceiling, so +/// it is what must fit), else the run's. +fn reserve_for(env: &StepEnv<'_>, i: usize) -> u64 { + llm_cfg(env, i) + .and_then(|c| c.max_tokens) + .map(u64::from) + .unwrap_or(env.llm_reserve_tokens) +} + +fn json_kind(v: &Value) -> &'static str { + match v { + Value::Null => "null", + Value::Bool(_) => "boolean", + Value::Number(_) => "number", + Value::String(_) => "string", + Value::Array(_) => "array", + Value::Object(_) => "object", + } +} + +/// `messages[0]`'s content: `{"instruction", "state"}`, shaped by the block. +fn first_message(env: &StepEnv<'_>, i: usize, input: &Value) -> Result { + let cfg = llm_cfg(env, i); + let instruction = match (cfg.and_then(|c| c.instruction.as_ref()), cfg.and_then(|c| c.instruction_from.as_ref())) { + (Some(text), _) => text.clone(), + (None, Some(ptr)) => match input.pointer(ptr) { + Some(Value::String(text)) => text.clone(), + Some(other) => { + return Err(format!( + "instruction_from {ptr:?} resolves to a {} in the node's input, not a string", + json_kind(other) + )) + } + None => return Err(format!("instruction_from {ptr:?} resolves to nothing in the node's input")), + }, + (None, None) => env.plan.nodes[i].clone(), + }; + let mut content = serde_json::Map::new(); + content.insert("instruction".into(), Value::String(instruction)); + // The tool list `tools_from` read is configuration, not state: under the + // whole-state default it would put every entry — and every value the host + // LOCKED away from the model — into the prompt. Removed from the copy the + // first message carries; a key list naming it is the plan's explicit ask. + let whole = cfg.and_then(|c| c.state.as_ref()).is_none_or(|s| s.as_str() == Some("all")); + let pruned; + let input = match cfg.and_then(|c| c.tools_from.as_deref()) { + Some(ptr) if whole && !ptr.is_empty() => { + let mut copy = input.clone(); + remove_pointer(&mut copy, ptr); + pruned = copy; + &pruned + } + _ => input, + }; + if let Some(state) = select_state(cfg.and_then(|c| c.state.as_ref()), input) { + content.insert("state".into(), state); + } + Ok(Value::Object(content)) +} + +/// Remove the value at a JSON pointer (RFC 6901, `~0`/`~1` unescaped), if any. +fn remove_pointer(v: &mut Value, ptr: &str) { + let Some((parent, last)) = ptr.rsplit_once('/') else { return }; + let last = last.replace("~1", "/").replace("~0", "~"); + let target = if parent.is_empty() { Some(v) } else { v.pointer_mut(parent) }; + match target { + Some(Value::Object(o)) => { + o.remove(&last); + } + Some(Value::Array(a)) => { + if let Ok(n) = last.parse::() { + if n < a.len() { + a.remove(n); + } + } + } + _ => {} + } +} + +/// The merged state a node's first message carries. `"all"` (or no block): +/// the whole input. `"none"`: nothing. A key list: those keys that are +/// present, nothing else. A steering `$inbox` rides along in every case — an +/// operator's message to the run is not state the plan may filter out. +fn select_state(sel: Option<&Value>, input: &Value) -> Option { + let keep = |keys: &[&str]| { + let mut o = serde_json::Map::new(); + if let Value::Object(src) = input { + for k in keys.iter().copied().chain([crate::types::INBOX]) { + if let Some(v) = src.get(k) { + o.insert(k.to_string(), v.clone()); + } + } + } + o + }; + match sel { + None => Some(input.clone()), + Some(Value::String(s)) if s == "all" => Some(input.clone()), + Some(Value::String(s)) if s == "none" => { + let o = keep(&[]); + (!o.is_empty()).then_some(Value::Object(o)) + } + Some(Value::Array(keys)) => { + let keys: Vec<&str> = keys.iter().filter_map(Value::as_str).collect(); + Some(Value::Object(keep(&keys))) + } + // The manifest refused every other shape at start; a hand-edited one + // gets the whole state rather than a guess. + Some(_) => Some(input.clone()), + } +} + +/// The wire name of a node's response schema: what OpenAI accepts +/// (`^[a-zA-Z0-9_-]{1,64}$`), derived from the node id. +fn response_name(node: &str) -> String { + let safe: String = node + .chars() + .map(|c| if c.is_ascii_alphanumeric() || c == '_' || c == '-' { c } else { '_' }) + .take(64) + .collect(); + if safe.is_empty() { "response".into() } else { safe } +} + +/// The request fields a node's block sets, as journaled on each of its turns +/// under `llm` — `response_format` already in the OpenAI-compatible wire +/// shape, so the journal shows what was asked for as it went out. A fold's +/// summarizer turn takes the node's system text, temperature and ceiling (a +/// GPT-5 node's fold must not 400 on a temperature either) but neither the +/// schema nor `parallel_tool_calls`: it answers in prose and is offered no +/// tools. `None` when nothing is set, so such a turn is byte-identical to one +/// from a node with no block. +fn request_block(env: &StepEnv<'_>, i: usize, fold: bool, instruction: &str) -> Option { + let c = llm_cfg(env, i)?; + let mut b = serde_json::Map::new(); + if let Some(s) = &c.system { + // OpenAI's `json_object` mode refuses (HTTP 400) a request in which + // no message says "JSON". The built-in system prompt does; a node's + // own may not, and neither may its instruction — so say it once. + let wants_json_object = !fold && c.response.as_ref().is_some_and(|r| r.get("json").is_some()); + let says_json = |t: &str| t.to_ascii_lowercase().contains("json"); + let s = if wants_json_object && !says_json(s) && !says_json(instruction) { + format!("{s} Answer with a JSON object.") + } else { + s.clone() + }; + b.insert("system".into(), Value::String(s)); + } + if let Some(t) = &c.temperature { + b.insert("temperature".into(), t.clone()); + } + if let Some(m) = c.max_tokens { + b.insert("max_tokens".into(), Value::from(m)); + } + if !fold { + if let Some(p) = c.parallel_tool_calls { + b.insert("parallel_tool_calls".into(), Value::Bool(p)); + } + if let Some(r) = &c.response { + let wire = match r.get("json_schema") { + Some(schema) => serde_json::json!({ + "type": "json_schema", + "json_schema": { + "name": response_name(&env.plan.nodes[i]), + "schema": schema, + "strict": true, + }, + }), + None => serde_json::json!({ "type": "json_object" }), + }; + b.insert("response_format".into(), wire); + } + } + (!b.is_empty()).then_some(Value::Object(b)) +} + +/// Check a final answer against the node's `response` declaration: a JSON +/// object, conforming to the schema when one is declared. The runtime checks +/// whatever the provider did with `response_format`, because a provider that +/// ignored the field must not hand prose through as the schema's object. +/// The answer inside a Markdown code fence, when the whole text is one — +/// ```` ```json … ``` ```` or ```` ``` … ``` ````. Anthropic models wrap a JSON +/// answer that way even when told not to. Anything else is returned trimmed. +fn unfence(text: &str) -> &str { + let t = text.trim(); + let Some(rest) = t.strip_prefix("```") else { return t }; + let Some(body) = rest.strip_suffix("```") else { return t }; + // The info string runs to the first newline (`json`, `JSON`, or nothing). + match body.split_once('\n') { + Some((info, inner)) if !info.contains('{') && !info.contains('[') => inner.trim(), + _ => body.trim(), + } +} + +fn check_response(response: &Value, text: &str) -> Result<(), String> { + let value: Value = serde_json::from_str(unfence(text)) + .map_err(|e| format!("the answer is not JSON ({e})"))?; + if !value.is_object() { + return Err(format!("the answer is a JSON {}, not an object", json_kind(&value))); + } + if let Some(schema) = response.get("json_schema") { + areev_core::types::validate_instance(&value, schema) + .map_err(|k| format!("schema violation: {}", k.as_str()))?; + } + Ok(()) +} + +/// Validate one model-issued call and build the input it dispatches with +/// (#392, #393). A dynamic tool's arguments are checked against ITS schema, +/// always — that is the contract the host declared for it. Then the locks are +/// merged, and the strict-argument table judges the MERGED input under the +/// Definition the call reaches, so a key the model was never shown (stripped +/// from its schema because locked) is not reported missing. +fn validated_dispatch_input( + env: &StepEnv<'_>, + tool: &OfferedTool, + arguments: &Value, +) -> Result { + if let Some(d) = &tool.dynamic { + // The entry's schema is CLOSED (#392 review): a key it does not name + // could otherwise set a Definition parameter the host never offered + // — `method: "DELETE"` on an http-shaped Definition that locks only + // the URL. A key the host LOCKED is refused outright rather than + // silently overridden, so the model learns it cannot choose it. + let schema = d.model_schema(tool.locked.as_ref()); + if let Value::Object(args) = arguments { + if d.dispatch.args_into.is_none() { + if let Some(Value::Object(lock)) = &tool.locked { + if let Some(k) = args.keys().find(|k| lock.contains_key(k.as_str())) { + return Err(format!( + "argument '{k}' is fixed by the host and may not be set" + )); + } + } + } + // Closed only where the arguments land at the Definition's TOP + // LEVEL (no `args_into`): there an unnamed key is a Definition + // parameter the host never offered. Wrapped under `args_into`, + // the arguments are the remote tool's own, and the entry's schema + // speaks for itself — an MCP tool's `{"type": "object"}` takes + // any key, and only `additionalProperties: false` closes it. + let closed = d.dispatch.args_into.is_none() + || schema.get("additionalProperties") == Some(&Value::Bool(false)); + let named = schema.get("properties").and_then(Value::as_object); + if closed { + if let Some(k) = args.keys().find(|k| !named.is_some_and(|p| p.contains_key(k.as_str()))) { + return Err(format!("argument '{k}' is not one of this tool's parameters")); + } + } + } + // Against what the model was SHOWN — locked keys stripped — so a + // locked key the entry also lists as required is not demanded of it. + areev_core::types::validate_instance(arguments, &schema) + .map_err(|k| format!("schema violation: {}", k.as_str()))?; + } + let input = tool.dispatch_input(arguments); + if let Some(schema) = tool.dynamic.as_ref().and_then(|d| d.dispatch.definition_schema.as_ref()) { + areev_core::types::validate_instance(&input, schema) + .map_err(|k| format!("schema violation: {} (the call as dispatched)", k.as_str()))?; + } + (env.validate_args)(tool.dispatch_name(), &input)?; + Ok(input) +} + fn flow_owners(st: &SchedulerState) -> Vec<(usize, String)> { let mut owners: Vec<(usize, String)> = st .abstract_flows @@ -656,7 +948,7 @@ fn handle_llm_outcome( path: &str, outcome: &EffectOutcome, ) { - let NodeExecutor::Abstract { tools } = &env.executors[i] else { return }; + let NodeExecutor::Abstract { tools, .. } = &env.executors[i] else { return }; let flow_id = flow_key(i, path); // A summarizer turn resolves differently from a model turn: its result is // not the node's answer, and any tool calls in it are ignored (it was @@ -802,6 +1094,9 @@ fn handle_llm_outcome( // Argument validation (§6.11): unparseable argument payloads and // strict-schema violations re-prompt with the error appended — // consuming `effect_seq`, bounded by `max_effects_per_attempt`. + // The check runs on the input the call will DISPATCH with — + // locks merged (#393), a dynamic tool's own schema first (#392) — + // so a strict Definition is judged on what it will receive. let invalid: Option<(String, String)> = calls.iter().find_map(|c| { if !c.arguments.is_object() { return Some(( @@ -809,7 +1104,8 @@ fn handle_llm_outcome( "arguments are not a JSON object".to_string(), )); } - (env.validate_args)(&c.tool_name, &c.arguments) + let tool = tools.iter().find(|t| t.tool_name == c.tool_name)?; + validated_dispatch_input(env, tool, &c.arguments) .err() .map(|e| (c.tool_name.clone(), e)) }); @@ -830,12 +1126,69 @@ fn handle_llm_outcome( // JSON object merges as one; anything else lands under the // node's own key. let final_text = text.unwrap_or_default(); - let value = serde_json::from_str::(&final_text) + // Structured output (#391): the answer must be what the block + // asked for. ONE corrective re-prompt, with the model's own + // answer kept in the transcript so it can see what it said, + // then the node fails — never a silent merge of prose. + let cfg = llm_cfg(env, i); + if let Some(response) = cfg.and_then(|c| c.response.as_ref()) { + if let Err(why) = check_response(response, &final_text) { + if flow.response_strikes >= 1 { + fail_abstract_coded( + env, + st, + i, + path, + &format!( + "SchemaValidationFailed: the final answer failed the node's \ + response declaration after a corrective re-prompt ({why})" + ), + ); + return; + } + flow.response_strikes += 1; + let mut entry = serde_json::json!({ + "role": "assistant", + "text": final_text, + "tool_calls": [], + }); + if let Some(pc) = provider_content { + entry["provider_content"] = pc; + } + flow.messages.push(entry); + flow.messages.push(serde_json::json!({ + "role": "user", + "content": format!( + "Your final answer failed validation: {why}. Answer again with \ + only a JSON object that satisfies the required response format." + ), + })); + flow.need = Some(crate::state::FlowNeed::NextTurn); + return; + } + } + // A declared response was checked on the unfenced text, so it + // merges from the same text; a node with no declaration merges + // exactly what it always did. + let merge_text = if cfg.is_some_and(|c| c.response.is_some()) { + unfence(&final_text).to_string() + } else { + final_text.clone() + }; + let mut value = serde_json::from_str::(&merge_text) .ok() .filter(|v| v.is_object()) .unwrap_or_else(|| { serde_json::json!({ env.plan.nodes[i].clone(): final_text }) }); + // What the model read, for whatever is bound below this node + // (#391): every tool call of the attempt, off the journaled + // outcomes. The runtime's key wins over one the model wrote. + if let Some(key) = cfg.and_then(|c| c.tool_results_into.as_ref()) { + if let Value::Object(o) = &mut value { + o.insert(key.clone(), Value::Array(std::mem::take(&mut flow.tool_log))); + } + } st.abstract_flows.remove(&flow_id); if path.is_empty() { st.node_state[i] = NodeState::DoneOk; @@ -1020,8 +1373,51 @@ fn handle_flow_tool_outcome( // truncation's journal pointer resolvable. let attempt = flow_attempt(st, i, path); let cap = env.llm_tool_result_chars; + let logging = llm_cfg(env, i).is_some_and(|c| c.tool_results_into.is_some()); + let journal_id = JournalKey { + run_id: st.run_id.clone(), + task_path: path.to_string(), + node: env.plan.nodes[i].clone(), + attempt, + effect_seq, + kind: EffectKind::Tool, + } + .tool_call_id(); let Some(flow) = st.abstract_flows.get_mut(&flow_key(i, path)) else { return }; let Some(call) = flow.pending_tools.remove(&effect_seq) else { return }; + if logging { + // `tool_results_into` (#391): the record a downstream gate reads. The + // input is what was DISPATCHED (locks merged), the result is the WHOLE + // journaled result — never the transcript's bounded excerpt — and the + // entry is placed by `effect_seq`, so arrival order cannot reorder it. + let NodeExecutor::Abstract { tools, .. } = &env.executors[i] else { return }; + let input = tools + .iter() + .find(|t| t.tool_name == call.tool_name) + .map(|t| t.dispatch_input(&call.arguments)) + .unwrap_or_else(|| call.arguments.clone()); + let (result, is_error) = match outcome { + EffectOutcome::Completed { result, .. } => (result.clone(), false), + EffectOutcome::Failed { cause, detail, .. } => { + (Value::String(format!("{cause:?}: {detail}")), true) + } + }; + let entry = serde_json::json!({ + "name": call.tool_name, + "tool_call_id": journal_id, + "model_call_id": call.model_call_id, + "input": input, + "result": result, + "is_error": is_error, + "effect_seq": effect_seq, + }); + let at = flow + .tool_log + .iter() + .position(|e| e.get("effect_seq").and_then(Value::as_u64).unwrap_or(0) > u64::from(effect_seq)) + .unwrap_or(flow.tool_log.len()); + flow.tool_log.insert(at, entry); + } let entry = match outcome { EffectOutcome::Completed { result, .. } => serde_json::json!({ "role": "tool", @@ -1496,10 +1892,13 @@ fn emit_fold_turn( effect_seq, kind: EffectKind::Llm, }; - let input = serde_json::json!({ + let mut input = serde_json::json!({ "messages": messages, "fold": { "from": from, "to": to, "seq": effect_seq, "prompt_v": FOLD_PROMPT_V }, }); + if let Some(block) = request_block(env, i, true, "") { + input["llm"] = block; + } emit_effect(env, st, i, path, key, input, out); } @@ -1515,9 +1914,10 @@ fn dispatch_llm_turn( out: &mut Vec, ) { let flow_id = flow_key(i, path); + let reserve = reserve_for(env, i); if let Some(max) = env.budgets.max_tokens { let spent = st.spent.input_tokens + st.spent.output_tokens; - if spent + env.llm_reserve_tokens > max { + if spent + reserve > max { st.exhausted = Some(crate::error::BudgetAxis::Tokens); // The un-dispatched turn survives as a need, so a resume with a // raised ceiling picks the loop up exactly here. @@ -1539,7 +1939,7 @@ fn dispatch_llm_turn( let projected = flow_now .map(|f| f.last_prompt_tokens) .unwrap_or(0) - .saturating_add(env.llm_reserve_tokens); + .saturating_add(reserve); let idle = flow_now.is_some_and(|f| f.folding.is_none()); // Two independent reasons to fold. The ceiling is a prediction and can // be unset; the provider's refusal is a fact and always wins. @@ -1613,6 +2013,17 @@ fn dispatch_llm_turn( if let Some(offer) = offer { input["offer"] = offer; } + // #391: the node's request fields ride the journaled input too, which is + // how the driver builds the request from the journal alone — and how a + // reader of the run sees the system text and response format as sent. + let instruction = messages + .first() + .and_then(|m| m.pointer("/content/instruction")) + .and_then(Value::as_str) + .unwrap_or_default(); + if let Some(block) = request_block(env, i, false, instruction) { + input["llm"] = block; + } emit_effect(env, st, i, path, key, input, out); } @@ -1661,7 +2072,7 @@ fn dispatch_flow_tool( // Resolve the offered tool BEFORE booking anything: a resolution set // that changed across a fork must fail the node loudly, not panic — // and never leave a booked pending tool with no outstanding key. - let NodeExecutor::Abstract { tools } = &env.executors[i] else { return }; + let NodeExecutor::Abstract { tools, .. } = &env.executors[i] else { return }; let Some(tool) = tools.iter().find(|t| t.tool_name == call.tool_name).cloned() else { fail_abstract( env, @@ -1672,10 +2083,29 @@ fn dispatch_flow_tool( ); return; }; + // The input it DISPATCHES with (#392, #393): locks merged over the + // model's arguments, a dynamic tool's wrapped first. Validated when the + // turn resolved; re-derived here because only the model's own arguments + // are kept in state (the transcript must show what the model said). A + // failure now means the offer changed under the call — a fork that + // migrated the plan — and fails the node loudly, as the missing-tool case + // above does. + let arguments = match validated_dispatch_input(env, &tool, &call.arguments) { + Ok(input) => input, + Err(why) => { + fail_abstract( + env, + st, + i, + path, + &format!("tool '{}' arguments no longer validate: {why}", call.tool_name), + ); + return; + } + }; let Some(flow) = st.abstract_flows.get_mut(&flow_id) else { return }; let effect_seq = flow.next_effect_seq; flow.next_effect_seq += 1; - let arguments = call.arguments.clone(); flow.pending_tools.insert(effect_seq, call.clone()); let key = JournalKey { run_id: st.run_id.clone(), @@ -1855,19 +2285,27 @@ fn emit_offer_decide( out: &mut Vec, ) -> bool { let Some(d) = env.decide.as_ref().filter(|d| d.calibrated) else { return false }; - let NodeExecutor::Abstract { tools } = &env.executors[i] else { return false }; + let NodeExecutor::Abstract { tools, .. } = &env.executors[i] else { return false }; + // The node's own rule (#396): `narrow: false` keeps its whole offer, + // `narrow: {top_k}` moves the K. + let Some(top_k) = narrow_top_k(env, i) else { return false }; // A choice carries 2..=255 options; past that the question cannot be // asked, and the full offer is the honest answer. - if tools.len() <= OFFER_TOP_K || tools.len() > 255 { + if tools.len() <= top_k || tools.len() > 255 { return false; } let Some(flow) = st.abstract_flows.get(&flow_key(i, path)) else { return false }; let first = flow.messages.first().and_then(|m| m.get("content")).cloned().unwrap_or(Value::Null); + // A dynamic tool (#392) is described by its OWN pinned description, its + // first line — exactly what a Definition's `tool_description` gives + // (#396). Read off the offer rather than the run-wide table, because two + // nodes may each declare a dynamic tool of one name. let describe = |t: &OfferedTool| { - d.tool_descriptions - .get(&t.tool_name) - .filter(|s| !s.trim().is_empty()) - .cloned() + t.dynamic + .as_ref() + .map(|dt| dt.description.lines().next().unwrap_or_default().trim().to_string()) + .filter(|s| !s.is_empty()) + .or_else(|| d.tool_descriptions.get(&t.tool_name).filter(|s| !s.trim().is_empty()).cloned()) .unwrap_or_else(|| t.tool_name.clone()) }; let criteria: serde_json::Map = @@ -2028,7 +2466,19 @@ fn handle_flow_decide( /// or above [`OFFER_KEEP_P`], returned in MANIFEST order. `None` when the /// table does not cover every pinned tool with a finite probability — a /// malformed answer narrows nothing. -fn narrow_offer(tools: &[OfferedTool], probs: &serde_json::Map) -> Option> { +/// The K node `i` narrows to, or `None` when its block says `narrow: false`. +fn narrow_top_k(env: &StepEnv<'_>, i: usize) -> Option { + match llm_cfg(env, i) { + Some(c) => c.narrow_top_k(OFFER_TOP_K), + None => Some(OFFER_TOP_K), + } +} + +fn narrow_offer( + tools: &[OfferedTool], + probs: &serde_json::Map, + top_k: usize, +) -> Option> { let mut ranked: Vec<(usize, f64)> = Vec::with_capacity(tools.len()); for (idx, t) in tools.iter().enumerate() { let p = probs.get(&t.tool_name).and_then(Value::as_f64).filter(|p| p.is_finite())?; @@ -2038,7 +2488,7 @@ fn narrow_offer(tools: &[OfferedTool], probs: &serde_json::Map) - let keep: std::collections::BTreeSet = ranked .iter() .enumerate() - .filter(|(rank, (_, p))| *rank < OFFER_TOP_K || *p >= OFFER_KEEP_P) + .filter(|(rank, (_, p))| *rank < top_k || *p >= OFFER_KEEP_P) .map(|(_, (idx, _))| *idx) .collect(); Some(keep.into_iter().map(|idx| tools[idx].tool_name.clone()).collect()) @@ -2052,7 +2502,7 @@ fn resolve_offer_decide( effect_seq: u32, outcome: &EffectOutcome, ) { - let NodeExecutor::Abstract { tools } = &env.executors[i] else { return }; + let NodeExecutor::Abstract { tools, .. } = &env.executors[i] else { return }; // Rule 2: only a CALIBRATED answer may omit a tool. Anything else — an // uncalibrated answer, a failure, a malformed table — keeps the full // offer, which is what the flow would have had with no backend. @@ -2063,7 +2513,7 @@ fn resolve_offer_decide( result .pointer("/answers/tool/probabilities") .and_then(Value::as_object) - .and_then(|probs| narrow_offer(tools, probs)) + .and_then(|probs| narrow_offer(tools, probs, narrow_top_k(env, i).unwrap_or(OFFER_TOP_K))) .filter(|keep| !keep.is_empty() && keep.len() < tools.len()) .map(|keep| { let mut record = decide_provenance(result); @@ -2803,6 +3253,30 @@ mod tests { // ---- the fold's range decision ------------------------------------- + #[test] + fn a_state_selection_keeps_the_steering_inbox_whatever_it_lists() { + let input = serde_json::json!({"a": 1, "b": 2, "$inbox": [{"text": "stop"}]}); + assert_eq!(select_state(None, &input), Some(input.clone())); + assert_eq!(select_state(Some(&serde_json::json!("all")), &input), Some(input.clone())); + assert_eq!( + select_state(Some(&serde_json::json!(["b", "zz"])), &input), + Some(serde_json::json!({"b": 2, "$inbox": [{"text": "stop"}]})) + ); + assert_eq!( + select_state(Some(&serde_json::json!("none")), &input), + Some(serde_json::json!({"$inbox": [{"text": "stop"}]})) + ); + assert_eq!(select_state(Some(&serde_json::json!("none")), &serde_json::json!({"a": 1})), None); + } + + #[test] + fn a_response_name_is_what_every_provider_accepts() { + assert_eq!(response_name("reply"), "reply"); + assert_eq!(response_name("ap.reply now"), "ap_reply_now"); + assert_eq!(response_name(&"n".repeat(80)).len(), 64); + assert_eq!(response_name(""), "response"); + } + fn msg(role: &str) -> Value { json!({"role": role, "content": "x"}) } diff --git a/crates/areev-run-core/src/types.rs b/crates/areev-run-core/src/types.rs index ae5dff65..bb54d0b7 100644 --- a/crates/areev-run-core/src/types.rs +++ b/crates/areev-run-core/src/types.rs @@ -212,6 +212,224 @@ pub enum EventIn { pub struct OfferedTool { pub tool_name: String, pub tool_hash: String, + /// Keys merged OVER the model's arguments at dispatch, the lock winning + /// (#393): the Definition's `locked_params`, and for a dynamic tool its + /// entry's `dispatch.locked` beneath them. What the intent journals and + /// the executor receives is the merged object; the transcript keeps what + /// the model said. `None` for every tool that locks nothing, so an offer + /// without locks serializes as it always did. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub locked: Option, + /// Present exactly when this is a tool declared in the run input (#392, + /// `tools_from`): the model sees `tool_name` and `dynamic.input_schema`, + /// and a call dispatches the pinned Definition `dynamic.dispatch` names — + /// whose address `tool_hash` is. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub dynamic: Option>, +} + +impl OfferedTool { + /// An offered pinned Definition with nothing locked — the pre-1.13 shape. + pub fn new(tool_name: impl Into, tool_hash: impl Into) -> Self { + OfferedTool { + tool_name: tool_name.into(), + tool_hash: tool_hash.into(), + locked: None, + dynamic: None, + } + } + + /// The Definition a call to this tool actually dispatches: its own name, + /// or — for a dynamic tool — the pinned Definition behind it. This is the + /// name the strict-argument table is keyed by. + pub fn dispatch_name(&self) -> &str { + match &self.dynamic { + Some(d) => &d.dispatch.definition, + None => &self.tool_name, + } + } + + /// The input a model-issued call dispatches with (#392, #393): the + /// model's arguments — under `dispatch.args_into` for a dynamic tool + /// that names one — with every locked key merged over them. Shallow, top + /// level, the lock always winning. Pure and total, so replay rebuilds the + /// same object; a non-object argument payload (refused before dispatch + /// anyway) passes through unlocked. + pub fn dispatch_input(&self, arguments: &Value) -> Value { + let mut input = match self.dynamic.as_ref().and_then(|d| d.dispatch.args_into.as_deref()) { + Some(key) => { + let mut o = serde_json::Map::new(); + o.insert(key.to_string(), arguments.clone()); + Value::Object(o) + } + None => arguments.clone(), + }; + if let (Some(Value::Object(lock)), Value::Object(obj)) = (&self.locked, &mut input) { + for (k, v) in lock { + obj.insert(k.clone(), v.clone()); + } + } + input + } +} + +/// Remove `locked`'s keys from a schema's `properties` and `required` (#393): +/// what the model is offered, and what its own arguments are checked against. +/// Pure, so the scheduler and the driver strip the same way. +pub fn strip_locked_keys(schema: &mut Value, locked: &Value) { + let Value::Object(lock) = locked else { return }; + if let Some(Value::Object(props)) = schema.get_mut("properties") { + for k in lock.keys() { + props.remove(k); + } + } + if let Some(Value::Array(req)) = schema.get_mut("required") { + req.retain(|r| r.as_str().is_none_or(|r| !lock.contains_key(r))); + } +} + +impl DynamicTool { + /// The schema the MODEL fills for this tool: the entry's `input_schema`, + /// minus the locked keys when the arguments land at the top level (with + /// `args_into` the locks belong to the Definition's input, one level up, + /// and say nothing about this schema). Offered to the model, and what its + /// arguments are validated against before the locks merge. + pub fn model_schema(&self, locked: Option<&Value>) -> Value { + let mut schema = self.input_schema.clone(); + if self.dispatch.args_into.is_none() { + if let Some(lock) = locked { + strip_locked_keys(&mut schema, lock); + } + } + schema + } +} + +/// A tool declared in the RUN INPUT and pinned at start (#392, `tools_from`). +/// +/// The host builds the list per run — the `tools/list` of an MCP server the +/// tenant connected, say — and the manifest freezes it, so a resume and +/// `verify` offer exactly the set the run started with. +#[derive(Debug, Clone, PartialEq, serde::Serialize, serde::Deserialize)] +pub struct DynamicTool { + /// What the model is offered and calls (normalized for providers like a + /// Definition's name: dots become underscores on the wire). + pub name: String, + #[serde(default, skip_serializing_if = "String::is_empty")] + pub description: String, + /// The arguments the model may fill; a call is validated against it, + /// always — it is the contract the host declared. + pub input_schema: Value, + pub dispatch: DynamicDispatch, +} + +/// Which pinned Definition a [`DynamicTool`] call reaches, and how. +#[derive(Debug, Clone, PartialEq, serde::Serialize, serde::Deserialize)] +pub struct DynamicDispatch { + /// The pinned host Definition's `tool_name` (canonical). + pub definition: String, + /// Its content address, frozen at start. + pub hash: String, + /// Keys merged over the model's arguments, the lock winning — the server + /// URL, the MCP tool name, the credential. The Definition's own + /// `locked_params` are merged over these in turn. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub locked: Option, + /// When set, the model's arguments are placed under this key before the + /// lock is merged — `"arguments"` for `mcp.call`, whose input carries the + /// MCP tool's arguments one level down. Absent = merged at the top level. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub args_into: Option, + /// The dispatch Definition's own `input_schema`, frozen at start. The + /// MERGED input — locks plus the wrapped arguments — is validated against + /// it whether or not the Definition is `strict`: the host promised the + /// model's arguments fit inside it, and a call that does not is a + /// correction for the model, not a request the Definition refuses. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub definition_schema: Option, +} + +/// A plan's per-node LLM configuration (#391, #392) — the `llm` block for one +/// abstract node, validated and normalized at start and FROZEN in the run +/// manifest, so a resume and `verify` run the block the run started with. +/// +/// Every field is optional and absent means "what an abstract node did before +/// the block existed": the built-in system prompt, the node name as the +/// instruction, the whole state, every pinned host tool, no structured output, +/// temperature 0, the run's `llm_max_tokens`. +#[derive(Debug, Clone, PartialEq, Default, serde::Serialize, serde::Deserialize)] +pub struct LlmNodeConfig { + /// The Definitions this node's dynamic tools may dispatch through that no + /// node binds (#392), by canonical name — pinned at start into the + /// manifest's `dispatch_pins`, never offered to the model themselves. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub dispatch: Vec, + /// Replaces the built-in system prompt for this node. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub system: Option, + /// The literal instruction placed in `messages[0]`. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub instruction: Option, + /// A JSON pointer into the node's input whose STRING is the instruction. + /// Resolving to nothing, or to a non-string, fails the node + /// `SchemaValidationFailed` (not retried — the same state fails the same + /// way), as a `reads` pointer does. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub instruction_from: Option, + /// Which merged-state keys `messages[0]` carries: `"all"` (the default), + /// `"none"`, or a list of keys. A steering `$inbox` is carried whatever + /// the selection says: an operator's message must reach the model. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub state: Option, + /// The subset of pinned host Definitions offered (by `tool_name`), in + /// manifest order. `None` = all of them. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub tools: Option>, + /// The JSON pointer the dynamic tools were read from, kept for the record. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub tools_from: Option, + /// The tools `tools_from` resolved to at start, in input order. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub dynamic_tools: Vec, + /// `{"json_schema": {…}}` or `{"json": true}`. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub response: Option, + /// A number, or `"omit"` to send none. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub temperature: Option, + /// This node's output ceiling per call, in place of the run's + /// `llm_max_tokens` — and its per-dispatch reservation. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub max_tokens: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub parallel_tool_calls: Option, + /// The node's result gains this key: every tool call of the attempt, from + /// the journaled outcomes. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub tool_results_into: Option, + /// The tool-offer narrowing for this node (C1, #396), normalized: + /// `false` keeps the whole offer, `{"top_k": n}` narrows to the top `n`. + /// `None` (also what `narrow: true` normalizes to) is the run-wide rule. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub narrow: Option, +} + +impl LlmNodeConfig { + /// `Some(k)` = narrow to the top `k` when more than `k` tools are offered; + /// `None` = never narrow this node. + pub fn narrow_top_k(&self, default: usize) -> Option { + match &self.narrow { + Some(Value::Bool(false)) => None, + Some(v) => Some( + v.get("top_k") + .and_then(Value::as_u64) + .map(|k| k as usize) + .filter(|k| *k >= 1) + .unwrap_or(default), + ), + None => Some(default), + } + } } /// What executes a node (resolved by the driver's V7 freeze). @@ -238,7 +456,13 @@ pub enum NodeExecutor { /// interprets, choosing among the workflow's pinned Host tools. The /// node runs as an LLM LOOP — turn, tool calls, results, next turn — /// each effect journaled under its own `effect_seq`. - Abstract { tools: Vec }, + Abstract { + tools: Vec, + /// The plan's frozen `llm` block for this node (#391), or `None` — + /// every abstract node before the block existed, byte for byte. + #[serde(default, skip_serializing_if = "Option::is_none")] + llm: Option>, + }, /// A subgraph node (Wave 2): the binding resolves to another Workflow /// grain; the child runs as its own run (own run_id, own journal), /// linked by `parent_task_id`, and its final context becomes this diff --git a/crates/areev-run-core/tests/scheduler_tests.rs b/crates/areev-run-core/tests/scheduler_tests.rs index 58ce3ec8..6234fa6d 100644 --- a/crates/areev-run-core/tests/scheduler_tests.rs +++ b/crates/areev-run-core/tests/scheduler_tests.rs @@ -55,13 +55,18 @@ struct SimResult { } impl Sim<'_> { - fn run(mut self) -> SimResult { + fn run(self) -> SimResult { + self.run_from(json!({"seed_input": true})) + } + + /// [`run`](Self::run) from a chosen run input. + fn run_from(mut self, input: Value) -> SimResult { let mut st = SchedulerState::new("run-1", self.env.plan); let mut all: Vec = Vec::new(); let mut checkpoints = Vec::new(); let mut events = vec![ EventIn::ClockReading { unix_ms: self.clock }, - EventIn::Start { input: json!({"seed_input": true}) }, + EventIn::Start { input }, ]; let mut guard = 0; loop { @@ -1025,8 +1030,9 @@ fn abstract_exec(names: &[&str]) -> Vec { vec![NodeExecutor::Abstract { tools: names .iter() - .map(|n| OfferedTool { tool_name: (*n).into(), tool_hash: "deadbeef".into() }) + .map(|n| OfferedTool::new(*n, "deadbeef")) .collect(), + llm: None, }] } @@ -1730,3 +1736,591 @@ fn a_decision_node_dispatches_like_a_host_tool_and_edges_branch_on_its_answer() let first = &dispatches(&r.commands)[0]; assert!(matches!(first.1, NodeExecutor::Decide { .. }) && first.0.kind == EffectKind::Tool); } + +// ---- the per-node `llm` block (#391), locked params (#393), dynamic tools (#392) + +fn llm_exec(tools: Vec, cfg: LlmNodeConfig) -> Vec { + vec![NodeExecutor::Abstract { tools, llm: Some(Box::new(cfg)) }] +} + +fn final_turn(text: &str) -> EffectOutcome { + ok(json!({"text": text, "tool_calls": [], "stop_reason": "end_turn"})) +} + +fn tool_turn(calls: Value) -> EffectOutcome { + ok(json!({"text": Value::Null, "tool_calls": calls, "stop_reason": "tool_use"})) +} + +fn intents(cmds: &[Command]) -> Vec<(JournalKey, NodeExecutor, Value)> { + cmds.iter() + .filter_map(|c| match c { + Command::WriteIntent { key, executor, input, .. } => { + Some((key.clone(), executor.clone(), input.clone())) + } + _ => None, + }) + .collect() +} + +#[test] +fn an_llm_block_shapes_the_first_message_and_rides_every_turn_input() { + let plan = PlanGraph::build(&wf(&["reply"])).unwrap(); + let cfg = LlmNodeConfig { + system: Some("You are the AP assistant.".into()), + instruction_from: Some("/prompt".into()), + state: Some(json!(["question"])), + temperature: Some(json!("omit")), + max_tokens: Some(512), + parallel_tool_calls: Some(false), + response: Some(json!({"json": true})), + ..Default::default() + }; + let execs = llm_exec(vec![OfferedTool::new("lookup", "h1")], cfg); + let behavior = |_k: &JournalKey, _i: &Value| final_turn(r#"{"answer": "paid"}"#); + let r = Sim { env: env(&plan, &execs, Budgets::default()), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run_from(json!({ + "prompt": "Was invoice 4471 paid?", + "question": "4471", + "account": {"huge": "x".repeat(50)}, + })); + assert_eq!(r.state.outcome(), Some(&RunOutcome::Completed)); + assert_eq!(r.state.context["answer"], json!("paid")); + let input = llm_input(&r.commands, 0); + assert_eq!( + input["messages"][0]["content"], + json!({"instruction": "Was invoice 4471 paid?", "state": {"question": "4471"}}), + "the instruction came from the pointer; only the listed key rode" + ); + assert_eq!( + input["llm"], + // json_object mode: the system text gains the word OpenAI requires. + json!({"system": "You are the AP assistant. Answer with a JSON object.", "temperature": "omit", "max_tokens": 512, + "parallel_tool_calls": false, "response_format": {"type": "json_object"}}), + "every request field the block sets is journaled as sent" + ); +} + +#[test] +fn an_empty_block_and_no_block_build_identical_commands() { + let plan = PlanGraph::build(&wf(&["reply"])).unwrap(); + let a = vec![NodeExecutor::Abstract { tools: vec![OfferedTool::new("t", "h")], llm: None }]; + let b = llm_exec(vec![OfferedTool::new("t", "h")], LlmNodeConfig::default()); + let behavior = |k: &JournalKey, _i: &Value| match (k.kind, k.effect_seq) { + (EffectKind::Llm, 0) => tool_turn(json!([{"id": "c0", "name": "t", "arguments": {"q": 1}}])), + (EffectKind::Llm, _) => final_turn("{\"done\": true}"), + (EffectKind::Tool, _) => ok(json!({"r": 1})), + }; + let run = |execs: &[NodeExecutor]| { + Sim { env: env(&plan, execs, Budgets::default()), behavior: &behavior, seed: 3, clock: 1_000, respond: BTreeMap::new() } + .run() + }; + let (ra, rb) = (run(&a), run(&b)); + assert_eq!( + serde_json::to_value(&ra.state).unwrap(), + serde_json::to_value(&rb.state).unwrap() + ); + assert_eq!(intents(&ra.commands).len(), intents(&rb.commands).len()); + for ((ka, _, ia), (kb, _, ib)) in intents(&ra.commands).into_iter().zip(intents(&rb.commands)) { + assert_eq!((ka, ia), (kb, ib)); + } +} + +#[test] +fn an_instruction_pointer_that_lands_on_nothing_fails_the_node_unretried_and_unsent() { + for (input, why) in [ + (json!({"other": 1}), "resolves to nothing"), + (json!({"prompt": 7}), "a number"), + ] { + let plan = PlanGraph::build(&wf(&["reply"]).retry("reply", 3)).unwrap(); + let cfg = LlmNodeConfig { instruction_from: Some("/prompt".into()), ..Default::default() }; + let execs = llm_exec(vec![], cfg); + let behavior = |_k: &JournalKey, _i: &Value| -> EffectOutcome { panic!("nothing may be dispatched") }; + let r = Sim { env: env(&plan, &execs, Budgets::default()), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run_from(input); + let Some(RunOutcome::Failed { node, detail }) = r.state.outcome() else { + panic!("{:?}", r.state.outcome()) + }; + assert_eq!(node, "reply"); + assert!(detail.starts_with("SchemaValidationFailed:") && detail.contains(why), "{detail}"); + assert!(intents(&r.commands).is_empty(), "no effect was written"); + assert_eq!(r.state.attempt[0], 1, "not retried: the same state fails the same way"); + } +} + +#[test] +fn state_none_sends_no_state_and_a_literal_instruction_wins() { + let plan = PlanGraph::build(&wf(&["reply"])).unwrap(); + let cfg = LlmNodeConfig { + instruction: Some("Say hello.".into()), + state: Some(json!("none")), + ..Default::default() + }; + let execs = llm_exec(vec![], cfg); + let behavior = |_k: &JournalKey, _i: &Value| final_turn("hello"); + let r = Sim { env: env(&plan, &execs, Budgets::default()), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run(); + let input = llm_input(&r.commands, 0); + assert_eq!(input["messages"][0]["content"], json!({"instruction": "Say hello."})); + assert!(input.get("llm").is_none(), "no request field set, so none journaled"); + assert_eq!(r.state.context["reply"], json!("hello"), "prose lands under the node id"); +} + +#[test] +fn a_response_schema_gets_one_corrective_reprompt_then_fails_the_node() { + let plan = PlanGraph::build(&wf(&["reply"])).unwrap(); + let schema = json!({"type": "object", "properties": {"paid": {"type": "boolean"}}, "required": ["paid"]}); + let cfg = LlmNodeConfig { response: Some(json!({"json_schema": schema})), ..Default::default() }; + let execs = llm_exec(vec![], cfg.clone()); + + // Prose, then a conforming object: one re-prompt, then it merges. + let behavior = |k: &JournalKey, _i: &Value| match k.effect_seq { + 0 => final_turn("It was paid."), + _ => final_turn(r#"{"paid": true}"#), + }; + let r = Sim { env: env(&plan, &execs, Budgets::default()), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run(); + assert_eq!(r.state.outcome(), Some(&RunOutcome::Completed)); + assert_eq!(r.state.context["paid"], json!(true)); + let second = llm_input(&r.commands, 1); + let msgs = second["messages"].as_array().unwrap(); + assert_eq!(msgs[1]["role"], "assistant", "the model sees what it answered"); + assert_eq!(msgs[1]["text"], "It was paid."); + assert!(msgs[2]["content"].as_str().unwrap().contains("failed validation"), "{msgs:?}"); + assert_eq!( + second["llm"]["response_format"]["json_schema"]["schema"], schema, + "the schema rides the request on the re-prompt too" + ); + assert_eq!(second["llm"]["response_format"]["json_schema"]["name"], "reply"); + + // A schema violation twice: the node fails, schema_validation_failed. + let behavior = |_k: &JournalKey, _i: &Value| final_turn(r#"{"paid": "yes"}"#); + let r = Sim { env: env(&plan, &execs, Budgets::default()), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run(); + let Some(RunOutcome::Failed { detail, .. }) = r.state.outcome() else { panic!("{:?}", r.state.outcome()) }; + assert!(detail.starts_with("SchemaValidationFailed:"), "{detail}"); + assert_eq!( + intents(&r.commands).iter().filter(|(k, _, _)| k.kind == EffectKind::Llm).count(), + 2, + "exactly one re-prompt" + ); +} + +#[test] +fn locked_params_override_the_model_and_the_strict_check_sees_the_merged_input() { + let plan = PlanGraph::build(&wf(&["agent"])).unwrap(); + let mut post = OfferedTool::new("post", "hpost"); + post.locked = Some(json!({"url": "https://a.example/x"})); + let execs = vec![NodeExecutor::Abstract { tools: vec![post], llm: None }]; + let seen: std::cell::RefCell> = Default::default(); + let validate = |tool: &str, args: &Value| -> std::result::Result<(), String> { + seen.borrow_mut().push(json!({"tool": tool, "args": args})); + // A strict schema requiring `url`, which the model was never shown. + if args.get("url").is_none() { Err("schema violation: required".into()) } else { Ok(()) } + }; + let behavior = |k: &JournalKey, _i: &Value| match (k.kind, k.effect_seq) { + (EffectKind::Llm, 0) => tool_turn(json!([{"id": "c0", "name": "post", + "arguments": {"url": "https://b.example/y", "body": "hi"}}])), + (EffectKind::Llm, _) => final_turn("{\"posted\": true}"), + (EffectKind::Tool, _) => ok(json!({"status": 200})), + }; + let r = Sim { + env: StepEnv { validate_args: &validate, ..env(&plan, &execs, Budgets::default()) }, + behavior: &behavior, + seed: 1, + clock: 1_000, + respond: BTreeMap::new(), + } + .run(); + assert_eq!(r.state.outcome(), Some(&RunOutcome::Completed)); + let (_, exec, input) = intents(&r.commands).into_iter().find(|(k, _, _)| k.kind == EffectKind::Tool).unwrap(); + assert_eq!(input, json!({"url": "https://a.example/x", "body": "hi"}), "the lock wins, and is journaled"); + assert_eq!(exec, NodeExecutor::Host { tool_hash: "hpost".into(), tool_name: "post".into() }); + assert_eq!(seen.borrow()[0]["args"]["url"], "https://a.example/x", "validated merged"); + // The transcript keeps what the model said. + let turn = llm_input(&r.commands, 2); + assert_eq!(turn["messages"][1]["tool_calls"][0]["arguments"]["url"], "https://b.example/y"); +} + +fn dynamic_paid_to() -> OfferedTool { + OfferedTool { + tool_name: "paid_to".into(), + tool_hash: "hmcp".into(), + locked: Some(json!({"url": "https://mcp.example/mcp", "tool": "paid_to", "credential": "gateway"})), + dynamic: Some(Box::new(DynamicTool { + name: "paid_to".into(), + description: "Who an invoice was paid to".into(), + input_schema: json!({"type": "object", + "properties": {"invoice": {"type": "string"}}, "required": ["invoice"], + "additionalProperties": false}), + dispatch: DynamicDispatch { + definition: "mcp.call".into(), + hash: "hmcp".into(), + locked: Some(json!({"url": "https://mcp.example/mcp", "tool": "paid_to", "credential": "gateway"})), + args_into: Some("arguments".into()), + definition_schema: None, + }, + })), + } +} + +#[test] +fn a_dynamic_tool_validates_its_own_schema_then_dispatches_the_pinned_definition() { + let plan = PlanGraph::build(&wf(&["agent"])).unwrap(); + let execs = vec![NodeExecutor::Abstract { tools: vec![dynamic_paid_to()], llm: None }]; + let strict_seen: std::cell::RefCell> = Default::default(); + let validate = |tool: &str, _args: &Value| -> std::result::Result<(), String> { + strict_seen.borrow_mut().push(tool.to_string()); + Ok(()) + }; + let behavior = |k: &JournalKey, _i: &Value| match (k.kind, k.effect_seq) { + // First a call that violates the dynamic schema (unknown key)… + (EffectKind::Llm, 0) => tool_turn(json!([{"id": "c0", "name": "paid_to", + "arguments": {"invoice": "4471", "url": "https://evil.example"}}])), + // …then a good one, which the model may also try to point elsewhere — + // the lock holds regardless. + (EffectKind::Llm, 1) => tool_turn(json!([{"id": "c1", "name": "paid_to", + "arguments": {"invoice": "4471"}}])), + (EffectKind::Llm, _) => final_turn("{\"done\": true}"), + (EffectKind::Tool, _) => ok(json!({"status": 200, "result": {"payee": "ACME"}})), + }; + let r = Sim { + env: StepEnv { validate_args: &validate, ..env(&plan, &execs, Budgets::default()) }, + behavior: &behavior, + seed: 1, + clock: 1_000, + respond: BTreeMap::new(), + } + .run(); + assert_eq!(r.state.outcome(), Some(&RunOutcome::Completed)); + let reprompt = llm_input(&r.commands, 1); + let last = reprompt["messages"].as_array().unwrap().last().unwrap().clone(); + assert!(last["content"].as_str().unwrap().contains("failed validation"), "{last}"); + let (_, exec, input) = intents(&r.commands).into_iter().find(|(k, _, _)| k.kind == EffectKind::Tool).unwrap(); + assert_eq!( + exec, + NodeExecutor::Host { tool_hash: "hmcp".into(), tool_name: "paid_to".into() }, + "journaled under the dynamic name, with the Definition's address" + ); + assert_eq!( + input, + json!({"url": "https://mcp.example/mcp", "tool": "paid_to", "credential": "gateway", + "arguments": {"invoice": "4471"}}) + ); + assert!( + strict_seen.borrow().iter().all(|t| t == "mcp.call"), + "the strict table is consulted under the dispatched Definition: {:?}", + strict_seen.borrow() + ); +} + +#[test] +fn tool_results_into_lists_every_call_in_effect_seq_order_from_the_outcomes() { + let plan = PlanGraph::build(&wf(&["agent"])).unwrap(); + let cfg = LlmNodeConfig { tool_results_into: Some("tool_results".into()), ..Default::default() }; + let mut post = OfferedTool::new("post", "hpost"); + post.locked = Some(json!({"url": "https://a.example/x"})); + let execs = llm_exec(vec![OfferedTool::new("lookup", "hl"), post], cfg); + let behavior = |k: &JournalKey, _i: &Value| match (k.kind, k.effect_seq) { + (EffectKind::Llm, 0) => tool_turn(json!([ + {"id": "c0", "name": "lookup", "arguments": {"q": "a"}}, + {"id": "c1", "name": "post", "arguments": {"body": "b"}}, + {"id": "c2", "name": "lookup", "arguments": {"q": "c"}}, + ])), + (EffectKind::Llm, _) => final_turn("{\"done\": true}"), + (EffectKind::Tool, 2) => EffectOutcome::Failed { + cause: FailCause::ExecutorError, + detail: "boom".into(), + journal_bytes: 4, + }, + (EffectKind::Tool, seq) => ok(json!({"seq": seq, "blob": "x".repeat(200)})), + }; + let mut reference: Option = None; + for seed in 1..=12 { + let r = Sim { + env: StepEnv { llm_tool_result_chars: Some(20), ..env(&plan, &execs, Budgets::default()) }, + behavior: &behavior, + seed, + clock: 1_000, + respond: BTreeMap::new(), + } + .run(); + assert_eq!(r.state.outcome(), Some(&RunOutcome::Completed)); + let log = r.state.context["tool_results"].clone(); + let seqs: Vec = log.as_array().unwrap().iter().map(|e| e["effect_seq"].as_u64().unwrap()).collect(); + assert_eq!(seqs, vec![1, 2, 3], "seed {seed}: canonical order, not arrival order"); + assert_eq!(log[0]["name"], "lookup"); + assert_eq!(log[0]["model_call_id"], "c0"); + assert_eq!(log[0]["result"]["blob"].as_str().unwrap().len(), 200, "the whole result, not the excerpt"); + assert_eq!(log[1]["input"], json!({"body": "b", "url": "https://a.example/x"}), "as dispatched"); + assert_eq!(log[1]["is_error"], true); + assert_eq!(log[1]["result"], "ExecutorError: boom"); + let key = JournalKey { + run_id: "run-1".into(), + task_path: String::new(), + node: "agent".into(), + attempt: 1, + effect_seq: 3, + kind: EffectKind::Tool, + }; + assert_eq!(log[2]["tool_call_id"], key.tool_call_id(), "joins the journaled grain"); + assert_eq!(r.state.context["done"], true); + match &reference { + None => reference = Some(log), + Some(l) => assert_eq!(&log, l, "seed {seed}"), + } + } +} + +#[test] +fn a_nodes_max_tokens_is_the_reservation_its_turns_must_fit() { + let plan = PlanGraph::build(&wf(&["reply"])).unwrap(); + let cfg = LlmNodeConfig { max_tokens: Some(4096), ..Default::default() }; + let execs = llm_exec(vec![], cfg); + let behavior = |_k: &JournalKey, _i: &Value| final_turn("hi"); + // The run's 1024 would fit a 2000-token budget; this node's 4096 does not. + let budgets = Budgets { max_tokens: Some(2_000), ..Budgets::default() }; + let r = Sim { env: env(&plan, &execs, budgets), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run(); + assert_eq!(r.state.outcome(), Some(&RunOutcome::BudgetExhausted { axis: BudgetAxis::Tokens })); + assert!(intents(&r.commands).is_empty()); +} + +// ---- #396: per-node narrowing, and dynamic tools described ---------------- + +fn twelve_with(cfg: LlmNodeConfig) -> Vec { + let tools = twelve_tools().into_iter().map(|n| OfferedTool::new(n, "deadbeef")).collect(); + llm_exec(tools, cfg) +} + +fn calibrated_offer(ask: &Value) -> EffectOutcome { + assert_eq!(ask["purpose"], "offer"); + ok(json!({ + "model": "jev-test", "provider": "fake", "calibrated": true, "latency_ms": 3, + "answers": {"tool": {"type": "choice", "choice": "t00", + "probabilities": offer_probabilities(), "confidence": 0.2}}, + })) +} + +#[test] +fn narrow_false_keeps_the_whole_offer_and_asks_nothing() { + let plan = PlanGraph::build(&wf(&["agent"])).unwrap(); + let execs = twelve_with(LlmNodeConfig { narrow: Some(json!(false)), ..Default::default() }); + let descs = BTreeMap::new(); + let script = || vec![llm_text(r#"{"done": true}"#, 100)]; + let with = scripted(script(), &no_decision); + let without = scripted(script(), &no_decision); + let a = run_with(decide_env(&plan, &execs, &descs, Some(true)), &with); + let b = run_with(decide_env(&plan, &execs, &descs, None), &without); + assert_eq!(a.state.outcome(), Some(&RunOutcome::Completed)); + assert!(decide_dispatches(&a.commands).is_empty(), "no narrowing ask"); + assert_eq!( + serde_json::to_string(&a.commands).unwrap(), + serde_json::to_string(&b.commands).unwrap(), + "the same run a host with no backend makes" + ); +} + +#[test] +fn narrow_top_k_moves_the_k_per_node() { + let plan = PlanGraph::build(&wf(&["agent"])).unwrap(); + let execs = twelve_with(LlmNodeConfig { narrow: Some(json!({"top_k": 3})), ..Default::default() }); + let descs = BTreeMap::new(); + let behavior = scripted(vec![llm_text(r#"{"done": true}"#, 100)], &calibrated_offer); + let r = run_with(decide_env(&plan, &execs, &descs, Some(true)), &behavior); + let turn = llm_input(&r.commands, 1); + let offered: Vec<&str> = + turn["offer"]["tools"].as_array().unwrap().iter().map(|v| v.as_str().unwrap()).collect(); + // Top 3, plus every tool at p ≥ 0.05 — the keep rule did not move. + assert_eq!(offered, vec!["t00", "t01", "t02", "t03", "t04", "t05", "t06", "t07", "t08"]); + + // And a K at or above the offer's size asks nothing at all. + let execs = twelve_with(LlmNodeConfig { narrow: Some(json!({"top_k": 12})), ..Default::default() }); + let behavior = scripted(vec![llm_text(r#"{"done": true}"#, 100)], &no_decision); + let r = run_with(decide_env(&plan, &execs, &descs, Some(true)), &behavior); + assert!(decide_dispatches(&r.commands).is_empty()); +} + +#[test] +fn a_narrowing_ask_describes_a_dynamic_tool_by_its_own_description() { + let plan = PlanGraph::build(&wf(&["agent"])).unwrap(); + let mut tools: Vec = + twelve_tools().into_iter().take(10).map(|n| OfferedTool::new(n, "deadbeef")).collect(); + let mut dynamic = dynamic_paid_to(); + dynamic.tool_name = "t10".into(); + if let Some(d) = dynamic.dynamic.as_mut() { + d.name = "t10".into(); + d.description = "Who an invoice was paid to\nsecond line is not context".into(); + } + tools.push(dynamic); + tools.push(OfferedTool::new("t11", "deadbeef")); + let execs = llm_exec(tools, LlmNodeConfig::default()); + let descs: BTreeMap = [("t00".to_string(), "does t00".to_string())].into(); + let seen: std::cell::RefCell> = Default::default(); + let decide = |ask: &Value| { + *seen.borrow_mut() = Some(ask.clone()); + calibrated_offer(ask) + }; + let behavior = scripted(vec![llm_text(r#"{"done": true}"#, 100)], &decide); + run_with(decide_env(&plan, &execs, &descs, Some(true)), &behavior); + let ask = seen.borrow().clone().expect("asked"); + assert_eq!(ask["questions"]["tool"]["criteria"]["t10"], "Who an invoice was paid to"); + assert_eq!(ask["questions"]["tool"]["criteria"]["t00"], "does t00", "a Definition's, as before"); + assert_eq!(ask["questions"]["tool"]["criteria"]["t11"], "t11", "no description: the name"); + let listed = ask["state"]["tools"].as_array().unwrap(); + assert!(listed.contains(&json!({"name": "t10", "description": "Who an invoice was paid to"}))); +} + +// ---- review fixes (#392/#393/#391) ------------------------------------------ + +/// An http-shaped dynamic entry with NO `args_into`: its arguments merge at +/// the top level of the Definition's input, with only the URL locked. +fn flat_entry() -> OfferedTool { + let lock = json!({"url": "https://a.example/x"}); + OfferedTool { + tool_name: "fetch_x".into(), + tool_hash: "hhttp".into(), + locked: Some(lock.clone()), + dynamic: Some(Box::new(DynamicTool { + name: "fetch_x".into(), + description: String::new(), + // `url` is required by the entry AND locked: the model is shown a + // schema without it and must not be asked for it. + input_schema: json!({"type": "object", + "properties": {"url": {"type": "string"}, "q": {"type": "string"}}, + "required": ["url", "q"]}), + dispatch: DynamicDispatch { + definition: "http.call".into(), + hash: "hhttp".into(), + locked: Some(lock), + args_into: None, + definition_schema: None, + }, + })), + } +} + +fn flat_run(first_args: Value) -> SimResult { + let plan = PlanGraph::build(&wf(&["agent"])).unwrap(); + let execs = vec![NodeExecutor::Abstract { tools: vec![flat_entry()], llm: None }]; + let behavior = move |k: &JournalKey, _i: &Value| match (k.kind, k.effect_seq) { + (EffectKind::Llm, 0) => tool_turn(json!([{"id": "c0", "name": "fetch_x", "arguments": first_args.clone()}])), + (EffectKind::Llm, 1) => tool_turn(json!([{"id": "c1", "name": "fetch_x", "arguments": {"q": "ok"}}])), + (EffectKind::Llm, _) => final_turn("{\"done\": true}"), + (EffectKind::Tool, _) => ok(json!({"status": 200})), + }; + Sim { env: env(&plan, &execs, Budgets::default()), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run() +} + +#[test] +fn a_flat_dynamic_entry_refuses_a_key_its_schema_does_not_name() { + let r = flat_run(json!({"q": "x", "method": "DELETE"})); + let reprompt = llm_input(&r.commands, 1); + let last = reprompt["messages"].as_array().unwrap().last().unwrap()["content"].as_str().unwrap().to_string(); + assert!(last.contains("'method' is not one of this tool's parameters"), "{last}"); + let dispatched: Vec = + dispatches(&r.commands).into_iter().filter(|(k, _, _)| k.kind == EffectKind::Tool).map(|(_, _, i)| i).collect(); + assert_eq!(dispatched, vec![json!({"q": "ok", "url": "https://a.example/x"})], "DELETE never went out"); +} + +#[test] +fn a_flat_dynamic_entry_refuses_a_locked_key_the_model_sends() { + let r = flat_run(json!({"q": "x", "url": "https://evil.example"})); + let reprompt = llm_input(&r.commands, 1); + let last = reprompt["messages"].as_array().unwrap().last().unwrap()["content"].as_str().unwrap().to_string(); + assert!(last.contains("'url' is fixed by the host"), "{last}"); +} + +#[test] +fn a_required_locked_key_is_not_demanded_of_the_model() { + // The first call omits `url` — exactly what the model was shown. + let r = flat_run(json!({"q": "first"})); + assert_eq!(r.state.outcome(), Some(&RunOutcome::Completed)); + let first_tool = dispatches(&r.commands).into_iter().find(|(k, _, _)| k.kind == EffectKind::Tool).unwrap(); + assert_eq!(first_tool.0.effect_seq, 1, "dispatched on the first turn, no re-prompt"); + assert_eq!(first_tool.2, json!({"q": "first", "url": "https://a.example/x"})); +} + +#[test] +fn json_object_mode_says_json_when_neither_the_system_nor_the_instruction_does() { + let plan = PlanGraph::build(&wf(&["reply"])).unwrap(); + let run = |cfg: LlmNodeConfig| { + let execs = llm_exec(vec![], cfg); + let behavior = |_k: &JournalKey, _i: &Value| final_turn("{\"a\": 1}"); + let r = Sim { env: env(&plan, &execs, Budgets::default()), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run(); + llm_input(&r.commands, 0)["llm"]["system"].as_str().unwrap().to_string() + }; + let json = || Some(json!({"json": true})); + assert_eq!( + run(LlmNodeConfig { system: Some("Be brief.".into()), response: json(), ..Default::default() }), + "Be brief. Answer with a JSON object." + ); + assert_eq!( + run(LlmNodeConfig { system: Some("Reply in json.".into()), response: json(), ..Default::default() }), + "Reply in json.", + "already says it" + ); + assert_eq!( + run(LlmNodeConfig { + system: Some("Be brief.".into()), + instruction: Some("Return a JSON summary".into()), + response: json(), + ..Default::default() + }), + "Be brief.", + "the instruction says it" + ); +} + +#[test] +fn a_fenced_answer_satisfies_a_response_declaration_and_merges() { + let plan = PlanGraph::build(&wf(&["reply"])).unwrap(); + let cfg = LlmNodeConfig { + response: Some(json!({"json_schema": {"type": "object", "properties": {"paid": {"type": "boolean"}}, + "required": ["paid"], "additionalProperties": false}})), + ..Default::default() + }; + let execs = llm_exec(vec![], cfg); + let behavior = |_k: &JournalKey, _i: &Value| final_turn("```json\n{\"paid\": true}\n```"); + let r = Sim { env: env(&plan, &execs, Budgets::default()), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run(); + assert_eq!(r.state.outcome(), Some(&RunOutcome::Completed)); + assert_eq!(r.state.context["paid"], json!(true)); + assert_eq!(intents(&r.commands).len(), 1, "no re-prompt"); +} + +#[test] +fn the_tools_from_list_stays_out_of_the_whole_state_prompt() { + let plan = PlanGraph::build(&wf(&["agent"])).unwrap(); + let cfg = LlmNodeConfig { tools_from: Some("/conn/tools".into()), ..Default::default() }; + let execs = llm_exec(vec![], cfg); + let behavior = |_k: &JournalKey, _i: &Value| final_turn("{}"); + let r = Sim { env: env(&plan, &execs, Budgets::default()), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run_from(json!({"ask": "q", "conn": {"tools": [{"locked": {"credential": "secret"}}], "org": "x"}})); + let first = &llm_input(&r.commands, 0)["messages"][0]["content"]["state"]; + assert_eq!(first, &json!({"ask": "q", "conn": {"org": "x"}})); +} + +#[test] +fn a_wrapped_entry_honours_its_own_schema_an_open_mcp_object_takes_any_key() { + // `args_into` set: the arguments are the MCP tool's, and its published + // `{"type": "object"}` names nothing and closes nothing. + let mut open = dynamic_paid_to(); + if let Some(d) = open.dynamic.as_mut() { + d.input_schema = json!({"type": "object"}); + } + let plan = PlanGraph::build(&wf(&["agent"])).unwrap(); + let execs = vec![NodeExecutor::Abstract { tools: vec![open], llm: None }]; + let behavior = |k: &JournalKey, _i: &Value| match (k.kind, k.effect_seq) { + (EffectKind::Llm, 0) => tool_turn(json!([{"id": "c0", "name": "paid_to", + "arguments": {"invoice": "4471", "year": 2026}}])), + (EffectKind::Llm, _) => final_turn("{}"), + (EffectKind::Tool, _) => ok(json!({"status": 200})), + }; + let r = Sim { env: env(&plan, &execs, Budgets::default()), behavior: &behavior, seed: 1, clock: 1_000, respond: BTreeMap::new() } + .run(); + let first_tool = dispatches(&r.commands).into_iter().find(|(k, _, _)| k.kind == EffectKind::Tool).unwrap(); + assert_eq!(first_tool.0.effect_seq, 1, "dispatched on the first turn"); + assert_eq!(first_tool.2["arguments"], json!({"invoice": "4471", "year": 2026})); +} diff --git a/crates/areev-run/CLAUDE.md b/crates/areev-run/CLAUDE.md index c2fb6976..b8ca32d5 100644 --- a/crates/areev-run/CLAUDE.md +++ b/crates/areev-run/CLAUDE.md @@ -428,6 +428,52 @@ The shape of the rules, and why each one is load-bearing: ceiling is also a two-backend case in `areev-conformance` (`run_recall_is_bounded_and_refused_past_its_ceiling`). +## The per-node `llm` block, `tools_from`, locks (`llmcfg.rs`, #391–#394) + +- **Frozen like `reads`.** `parse_llm` (called from `resolve_with_fields` after + the pins resolve) validates the plan's `llm` table: unknown keys and shapes + are `RUN-E019`, and a `tools` or `dispatch` name the run cannot resolve is + `RUN-E004`. The block becomes `PinnedTool.llm` on the abstract node's pin, + with `tools_from` resolved against the run INPUT there, once. Resume never + re-reads the plan or the input. +- **`llm.dispatch` pins live in `RunManifest.dispatch_pins`**, not in + `pinned`. `pinned` is one entry per plan node and `executors()` indexes it + by node. A dispatch pin resolves like an unbound node's Definition (the run + namespace, then the plan's), is checked at start through `code_pins()`, is + found at dispatch through `host_pin_by_hash`, joins the strict table, and is + never in any offer. +- **The request is built from the JOURNAL.** The scheduler writes the block's + request fields to each turn's `input.llm`. `TurnRequest::from_input` reads + them back for the request and for the span attributes, the same rule as + `input.fold` and `input.offer`. Never read them from the manifest at + prepare time. +- **A flow tool finds its Definition by frozen hash** (`host_pin_by_hash`), + never by node. The by-node lookup found the abstract node's own pin, so a + model's call to a code-carrying Definition ran `--tool-cmd`. A dynamic + tool's Host executor carries the DYNAMIC name (journal) and the + Definition's hash. `DispatchJob.exec_name` makes the pool execute it under + the Definition's name, which is what `--tool-cmd`, `register_capability` and + the broker's grants key on. Result grains still re-state the dispatched + executor, so intent and result agree on the name. +- **Locks.** `PinnedTool.locked_params` is frozen from the Definition, which + must be a host tool. `offered_definition` strips locked keys from the + offered schema, except for a dynamic entry with `args_into`, whose locks + belong to the Definition's input and not to the model's schema. The merge + itself is pure and happens in run-core (`OfferedTool::dispatch_input`). + `offered_definition` and run-core's validator share + `DynamicTool::model_schema`, so what the model is shown is what its + arguments are judged by. +- **`shadow_plan` takes locks from the INCUMBENT's pins** (by hash), never + from re-reading the Definition: a 1.12 run froze none, and rehearsing it + with them diverges on an unchanged plan. +- **Warnings ride the manifest** (`RunManifest.warnings`, `RUN-Wnnn …`), so + `run inspect` reports them. `RUN-W001` covers a client or decision + Definition carrying `locked_params`: ignored, as in 1.12, never refused. +- **Pseudonymization:** `llm.system` is read off `to_model`, and a dynamic + tool's prose goes through `pseudonymize_tool_text`. Structural schema + strings are left alone. + Tests: `tests/llm_node_tests.rs`. + ## Telemetry (`stream.rs` + `otel.rs`) **The event is the only channel.** The §6.10 observer runs on the bus's own diff --git a/crates/areev-run/Cargo.toml b/crates/areev-run/Cargo.toml index 1be48542..8fe6ff02 100644 --- a/crates/areev-run/Cargo.toml +++ b/crates/areev-run/Cargo.toml @@ -17,10 +17,10 @@ keywords = ["ai-agents", "workflow", "runtime", "deterministic", "areev"] categories = ["development-tools"] [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } -areev-store = { path = "../areev-store", version = "1.12.0" } -areev-cal = { path = "../areev-cal", version = "1.12.0" } -areev-run-core = { path = "../areev-run-core", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } +areev-store = { path = "../areev-store", version = "1.13.0" } +areev-cal = { path = "../areev-cal", version = "1.13.0" } +areev-run-core = { path = "../areev-run-core", version = "1.13.0" } # The ToolCallLlm seam for abstract nodes (§6.2/§6.11). # # `default-features = false` (#289): this crate names only seam TYPES @@ -35,7 +35,7 @@ areev-run-core = { path = "../areev-run-core", version = "1.12.0" } # # The hosts that DO call the factory (areev-cli, areev-py, areev-js, # areev-bench) depend on areev-llm directly and keep their defaults. -areev-llm = { path = "../areev-llm", version = "1.12.0", default-features = false } +areev-llm = { path = "../areev-llm", version = "1.13.0", default-features = false } # The egress broker's outbound leg (moved here from areev-trigger, which # depends on this crate and so could not share it). ureq = "3" diff --git a/crates/areev-run/src/executor.rs b/crates/areev-run/src/executor.rs index 8e9caf6c..d7063523 100644 --- a/crates/areev-run/src/executor.rs +++ b/crates/areev-run/src/executor.rs @@ -968,8 +968,24 @@ pub struct PreparedLlm { /// non-streaming call. Purely observational: the journaled result is /// the same either way. pub on_token: Option, + /// The node's own system text (#391); `None` = the built-in prompt. + pub system: Option, + /// The temperature to send; `None` sends none (`"temperature": "omit"`). + pub temperature: Option, + pub response_format: Option, + pub parallel_tool_calls: Option, + /// Record the temperature that actually went out on the result grain + /// (`temperature_sent`) — set when the turn's intent claims one (a node + /// `llm` block) or the transport's profile may change it. Unset for every + /// other turn, whose result stays byte-identical to 1.12's. + pub record_sent: bool, } +/// The system prompt an abstract node runs under when its plan sets none. +pub const DEFAULT_SYSTEM_PROMPT: &str = "You are executing one node of a governed workflow. Use \ +the offered tools when needed; when done, answer with the node's final result (a JSON object \ +when structured output is wanted)."; + /// One in-flight dispatch handed to the pool. pub struct DispatchJob { pub key: JournalKey, @@ -982,6 +998,12 @@ pub struct DispatchJob { /// Read and hash-verified on the driver thread, like `llm`'s pinned /// Definitions — the pool never touches a store handle. pub code: Option, + /// The name the host executor runs this effect under, when it is not the + /// executor's own: a dynamic tool (#392) is journaled under the name the + /// model called and EXECUTES as the pinned Definition it dispatches to — + /// the name `--tool-cmd`, a registry and the broker's grants know. + /// `None` = `executor`'s name, every effect before 1.13. + pub exec_name: Option, } /// A code blob, read off the driver thread and ready to execute. @@ -1073,7 +1095,7 @@ impl Pool { let (tool_name, tool_hash) = match &job.executor { NodeExecutor::Host { tool_name, tool_hash } | NodeExecutor::Client { tool_name, tool_hash, .. } => { - (tool_name.clone(), tool_hash.clone()) + (job.exec_name.clone().unwrap_or_else(|| tool_name.clone()), tool_hash.clone()) } NodeExecutor::Subgraph { workflow_hash } => { // Subgraphs never reach the pool — the driver runs @@ -1197,18 +1219,18 @@ fn run_llm_effect( journal_bytes: 0, }; }; + let record_sent = prepared.record_sent; + let sent_temperature = prepared.temperature.and_then(|t| llm.effective_temperature(t)); let request = areev_llm::ToolCallRequest { - system: Some( - "You are executing one node of a governed workflow. Use the \ - offered tools when needed; when done, answer with the node's \ - final result (a JSON object when structured output is wanted)." - .into(), - ), + system: Some(prepared.system.unwrap_or_else(|| DEFAULT_SYSTEM_PROMPT.to_string())), messages: prepared.messages, tools: &prepared.tools, tool_choice: areev_llm::ToolChoice::Auto, max_tokens: prepared.max_tokens, - temperature: 0.0, + temperature: prepared.temperature.unwrap_or(0.0), + omit_temperature: prepared.temperature.is_none(), + response_format: prepared.response_format, + parallel_tool_calls: prepared.parallel_tool_calls, }; let result = match prepared.on_token { Some(mut sink) => llm.call_streaming(&request, &mut *sink), @@ -1252,6 +1274,12 @@ fn run_llm_effect( if let Some(r) = &resp.served_region { result["served_region"] = serde_json::json!(r); } + // What went out, not what was asked (#394 review): the intent's + // `llm.temperature` is the plan's request, and a profile may have + // pinned or dropped it on the wire. + if record_sent { + result["temperature_sent"] = serde_json::json!(sent_temperature); + } // #291: the transport prices its own usage, or says it cannot. // `None` is UNPRICED, not free — `priced` is what keeps the two // apart downstream, so a cost bound over an unpriced run reads diff --git a/crates/areev-run/src/lib.rs b/crates/areev-run/src/lib.rs index 5b1da525..d5f4d064 100644 --- a/crates/areev-run/src/lib.rs +++ b/crates/areev-run/src/lib.rs @@ -22,6 +22,7 @@ pub mod egress_spec; pub mod executor; pub mod journal; pub mod lease; +pub mod llmcfg; pub mod manifest; pub mod memread; pub mod otel; @@ -163,6 +164,10 @@ pub struct InspectReport { /// Absent otherwise, so a report for a run nobody paused is unchanged. #[serde(skip_serializing_if = "Option::is_none")] pub pause: Option, + /// Start-time warnings the run was started under (`RUN-W001` …), from + /// the manifest. Absent when there were none. + #[serde(skip_serializing_if = "Vec::is_empty")] + pub warnings: Vec, } /// What `Runner::pause` answers (#344): the standing request, whether this diff --git a/crates/areev-run/src/llmcfg.rs b/crates/areev-run/src/llmcfg.rs new file mode 100644 index 00000000..51473560 --- /dev/null +++ b/crates/areev-run/src/llmcfg.rs @@ -0,0 +1,617 @@ +//! The per-node `llm` block of a plan (#391) and the dynamic tools a node +//! reads from the run input (`tools_from`, #392): validated in full at run +//! start, normalized, and FROZEN into the abstract node's pin in the run +//! manifest — exactly as `reads` are — so a superseded plan, or a host that +//! hands a resume a different input, cannot change what a running run sends. +//! +//! ```json +//! "llm": { +//! "reply": { +//! "system": "You are …", +//! "instruction_from": "/prompt", +//! "state": ["question", "account"], +//! "tools": ["records_query"], +//! "dispatch": ["mcp_call"], +//! "tools_from": "/tools", +//! "response": {"json_schema": {…}}, +//! "temperature": "omit", +//! "max_tokens": 2048, +//! "parallel_tool_calls": false, +//! "tool_results_into": "tool_results" +//! } +//! } +//! ``` +//! +//! Refusals name the node (and, for a dynamic tool, the entry): a shape +//! defect is `RUN-E019`; a tool name or a dispatch Definition the plan did not +//! pin is `RUN-E004`, the unresolved-reference code a binding gets. + +use crate::manifest::PinnedTool; +use areev_core::format::tool_schema::normalize_tool_name; +use areev_core::types::json_schema_subset::{is_valid_tool_name, strip_annotations, SchemaValidator}; +use areev_run_core::{DynamicDispatch, DynamicTool, LlmNodeConfig, PlanGraph, RunError}; +use serde_json::Value; +use std::collections::BTreeMap; + +/// The plan field the block lives under. +pub const LLM_FIELD: &str = "llm"; + +const BLOCK_KEYS: &[&str] = &[ + "dispatch", + "system", + "instruction", + "instruction_from", + "state", + "tools", + "tools_from", + "response", + "temperature", + "max_tokens", + "parallel_tool_calls", + "tool_results_into", + "narrow", +]; + +const ENTRY_KEYS: &[&str] = &["name", "description", "input_schema", "dispatch"]; +const DISPATCH_KEYS: &[&str] = &["definition", "locked", "args_into"]; + +/// Validate the plan's `llm` table against the resolved pins and the run +/// input, and return node → frozen block. Every node named must be in the +/// plan and resolve ABSTRACT: the block configures a model loop, and a node +/// that binds a tool runs no model to configure. +/// +/// `resolve` pins one `dispatch` entry (a Definition name or content address) +/// for a node — the store-side half, which this module does not hold. The +/// pins it returns come back beside the blocks, for the manifest's +/// `dispatch_pins`: pinned like a bound Definition (code address, runtime, +/// capabilities, locks), never offered to a model on their own. +pub fn parse_llm( + field: Option<&Value>, + plan: &PlanGraph, + pinned: &[PinnedTool], + input: &Value, + resolve: &mut dyn FnMut(&str, &str) -> Result, +) -> Result<(BTreeMap, Vec), RunError> { + let invalid = |why: String| RunError::InvalidPlan { why }; + let table = match field { + None | Some(Value::Null) => return Ok((BTreeMap::new(), Vec::new())), + Some(Value::Object(t)) => t, + Some(_) => return Err(invalid(format!("`{LLM_FIELD}` must be an object of node → block"))), + }; + // The universe a node's `tools` subset and a dynamic tool's dispatch are + // drawn from: the plan's pinned HOST Definitions, as an abstract node is + // offered them (decision nodes, reads, clients and subgraphs are never + // model-callable). + let host: Vec<&PinnedTool> = pinned.iter().filter(|p| p.executor == "host").collect(); + let mut out = BTreeMap::new(); + let mut dispatch_pins: Vec = Vec::new(); + for (node, raw) in table { + if !plan.nodes.iter().any(|n| n == node) { + return Err(invalid(format!("`{LLM_FIELD}` names unknown node '{node}'"))); + } + let pin = pinned.iter().find(|p| &p.node == node); + if pin.map(|p| p.executor.as_str()) != Some("abstract") { + return Err(invalid(format!( + "`{LLM_FIELD}` configures node '{node}', which resolves to a {} executor — the \ + block configures an abstract (LLM) node, and this one runs no model", + pin.map(|p| p.executor.as_str()).unwrap_or("missing") + ))); + } + // `dispatch` (#392): Definitions this node's dynamic tools may reach + // that no node binds — the blessed `mcp.call`, typically. Resolved + // first, so the entries below can name them. + let mut mine: Vec = Vec::new(); + match raw.get("dispatch") { + None | Some(Value::Null) => {} + Some(Value::Array(list)) => { + for e in list { + let Some(entry) = e.as_str().filter(|s| !s.is_empty()) else { + return Err(invalid(format!( + "node '{node}' llm: `dispatch` lists {e}, which is not a Definition \ + name or content address" + ))); + }; + let pin = resolve(node, entry)?; + if !mine.iter().any(|p| p.tool_hash == pin.tool_hash) { + mine.push(pin); + } + } + } + Some(other) => { + return Err(invalid(format!( + "node '{node}' llm: `dispatch` must be a list of Definition names or \ + content addresses, not {other}" + ))) + } + } + let mut universe: Vec<&PinnedTool> = mine.iter().collect(); + universe.extend(host.iter().copied()); + let mut cfg = parse_block(node, raw, &host, &universe, input)?; + cfg.dispatch = mine.iter().map(|p| p.tool_name.clone()).collect(); + out.insert(node.clone(), cfg); + dispatch_pins.extend(mine); + } + Ok((out, dispatch_pins)) +} + +fn parse_block( + node: &str, + raw: &Value, + host: &[&PinnedTool], + dispatchable: &[&PinnedTool], + input: &Value, +) -> Result { + let bad = |why: String| RunError::InvalidPlan { why: format!("node '{node}' llm: {why}") }; + let Value::Object(b) = raw else { + return Err(bad("the block must be an object".into())); + }; + if let Some(k) = b.keys().find(|k| !BLOCK_KEYS.contains(&k.as_str())) { + return Err(bad(format!("unknown key {k:?} (accepted: {})", BLOCK_KEYS.join(", ")))); + } + let string = |k: &str| -> Result, RunError> { + match b.get(k) { + None | Some(Value::Null) => Ok(None), + Some(Value::String(s)) => Ok(Some(s.clone())), + Some(other) => Err(bad(format!("`{k}` must be a string, not {other}"))), + } + }; + let mut cfg = LlmNodeConfig { + system: string("system")?, + instruction: string("instruction")?, + instruction_from: string("instruction_from")?, + ..Default::default() + }; + if cfg.instruction.is_some() && cfg.instruction_from.is_some() { + return Err(bad("`instruction` and `instruction_from` are exclusive — name one".into())); + } + if let Some(p) = &cfg.instruction_from { + check_pointer(p).map_err(|why| bad(format!("`instruction_from` {why}")))?; + } + + cfg.state = match b.get("state") { + None | Some(Value::Null) => None, + Some(Value::String(s)) if s == "all" || s == "none" => Some(Value::String(s.clone())), + Some(Value::Array(keys)) + if keys.iter().all(|k| k.as_str().is_some_and(|s| !s.is_empty())) => + { + Some(Value::Array(keys.clone())) + } + Some(other) => { + return Err(bad(format!( + "`state` must be \"all\", \"none\" or a list of state keys, not {other}" + ))) + } + }; + + // `tools`: a subset of the pinned host Definitions, by canonical name or + // the provider-safe spelling the model sees; stored canonical, in + // manifest order, once each. + cfg.tools = match b.get("tools") { + None | Some(Value::Null) => None, + Some(Value::Array(names)) => { + let mut picked: Vec = Vec::new(); + for n in names { + let Some(name) = n.as_str() else { + return Err(bad(format!("`tools` lists {n}, which is not a tool name"))); + }; + let Some(p) = host + .iter() + .find(|p| p.tool_name == name) + .or_else(|| host.iter().find(|p| normalize_tool_name(&p.tool_name) == name)) + else { + return Err(RunError::UnresolvedRef { + what: format!( + "node '{node}' llm.tools names '{name}', which is not one of the \ + plan's pinned host tools ({})", + pinned_names(host) + ), + }); + }; + if !picked.contains(&p.tool_name) { + picked.push(p.tool_name.clone()); + } + } + let mut ordered: Vec = Vec::new(); + for p in host { + if picked.contains(&p.tool_name) && !ordered.contains(&p.tool_name) { + ordered.push(p.tool_name.clone()); + } + } + Some(ordered) + } + Some(other) => return Err(bad(format!("`tools` must be a list of tool names, not {other}"))), + }; + + if let Some(ptr) = string("tools_from")? { + check_pointer(&ptr).map_err(|why| bad(format!("`tools_from` {why}")))?; + cfg.dynamic_tools = parse_dynamic(node, &ptr, input, host, dispatchable)?; + cfg.tools_from = Some(ptr); + } + + cfg.response = match b.get("response") { + None | Some(Value::Null) => None, + Some(Value::Object(r)) if r.len() == 1 && r.get("json") == Some(&Value::Bool(true)) => { + Some(serde_json::json!({"json": true})) + } + Some(Value::Object(r)) if r.len() == 1 && r.contains_key("json_schema") => { + let mut schema = r["json_schema"].clone(); + strip_annotations(&mut schema); + SchemaValidator::tool_schema().validate(&schema).map_err(|e| { + bad(format!("`response.json_schema` is not a usable schema: {e}")) + })?; + if let Some(why) = strict_mode_violation(&schema, "") { + return Err(bad(format!( + "`response.json_schema` is sent in strict mode, which needs every object \ + closed and every property required: {why}" + ))); + } + Some(serde_json::json!({"json_schema": schema})) + } + Some(other) => { + return Err(bad(format!( + "`response` must be {{\"json_schema\": {{…}}}} or {{\"json\": true}}, not {other}" + ))) + } + }; + + cfg.temperature = match b.get("temperature") { + None | Some(Value::Null) => None, + Some(Value::String(s)) if s == "omit" => Some(Value::String("omit".into())), + Some(Value::Number(n)) if n.as_f64().is_some_and(|t| t.is_finite() && t >= 0.0) => { + Some(Value::Number(n.clone())) + } + Some(other) => { + return Err(bad(format!( + "`temperature` must be a non-negative number or \"omit\", not {other}" + ))) + } + }; + + cfg.max_tokens = match b.get("max_tokens") { + None | Some(Value::Null) => None, + Some(v) => match v.as_u64().filter(|n| *n >= 1).and_then(|n| u32::try_from(n).ok()) { + Some(n) => Some(n), + None => { + return Err(bad(format!( + "`max_tokens` must be a positive integer (at most {}), not {v}", + u32::MAX + ))) + } + }, + }; + + cfg.parallel_tool_calls = match b.get("parallel_tool_calls") { + None | Some(Value::Null) => None, + Some(Value::Bool(p)) => Some(*p), + Some(other) => return Err(bad(format!("`parallel_tool_calls` must be a boolean, not {other}"))), + }; + + cfg.tool_results_into = match string("tool_results_into")? { + None => None, + Some(k) if !k.is_empty() && !k.starts_with('$') => Some(k), + Some(k) => { + return Err(bad(format!( + "`tool_results_into` must be a non-empty state key not starting with '$', got {k:?}" + ))) + } + }; + // `narrow` (#396): whether, and to how many, a calibrated decision + // backend narrows this node's offer. `true` is the run-wide rule and is + // stored as nothing, so it changes no manifest's bytes. + cfg.narrow = match b.get("narrow") { + None | Some(Value::Null) | Some(Value::Bool(true)) => None, + Some(Value::Bool(false)) => Some(Value::Bool(false)), + Some(Value::Object(o)) + if o.len() == 1 + && o.get("top_k").and_then(Value::as_u64).is_some_and(|k| (1..=254).contains(&k)) => + { + Some(serde_json::json!({"top_k": o["top_k"]})) + } + Some(other) => { + return Err(bad(format!( + "`narrow` must be true, false or {{\"top_k\": 1-254}}, not {other}" + ))) + } + }; + Ok(cfg) +} + +/// A JSON pointer as RFC 6901 writes one: empty (the whole document) or +/// starting with `/`. +fn check_pointer(p: &str) -> Result<(), String> { + if p.is_empty() || p.starts_with('/') { + Ok(()) + } else { + Err(format!("{p:?} is not a JSON pointer (it must start with '/')")) + } +} + +fn pinned_names(host: &[&PinnedTool]) -> String { + let mut names: Vec<&str> = host.iter().map(|p| p.tool_name.as_str()).collect(); + names.dedup(); + if names.is_empty() { + "the plan pins none".into() + } else { + format!("pinned: {}", names.join(", ")) + } +} + +/// Resolve `tools_from` against the run input and pin every entry. +/// +/// `host` is what a dynamic name must not collide with (the tools a model may +/// be offered); `dispatchable` is what an entry may dispatch through — the +/// node's `dispatch` pins, then the plan's host pins. +fn parse_dynamic( + node: &str, + ptr: &str, + input: &Value, + host: &[&PinnedTool], + dispatchable: &[&PinnedTool], +) -> Result, RunError> { + let bad = |why: String| RunError::InvalidPlan { why: format!("node '{node}' llm.tools_from: {why}") }; + let list = match input.pointer(ptr) { + Some(Value::Array(list)) => list, + Some(other) => { + return Err(bad(format!("{ptr:?} resolves to {other} in the run input, not a list of tools"))) + } + None => return Err(bad(format!("{ptr:?} resolves to nothing in the run input"))), + }; + let mut out: Vec = Vec::with_capacity(list.len()); + for (n, entry) in list.iter().enumerate() { + let label = entry + .get("name") + .and_then(Value::as_str) + .map(|s| format!("entry {n} ('{s}')")) + .unwrap_or_else(|| format!("entry {n}")); + let ebad = |why: String| bad(format!("{label}: {why}")); + let Value::Object(e) = entry else { + return Err(ebad("must be an object".into())); + }; + if let Some(k) = e.keys().find(|k| !ENTRY_KEYS.contains(&k.as_str())) { + return Err(ebad(format!("unknown key {k:?} (accepted: {})", ENTRY_KEYS.join(", ")))); + } + let name = match e.get("name").and_then(Value::as_str) { + Some(s) if is_valid_tool_name(s) => s.to_string(), + _ => { + return Err(ebad( + "`name` must be a valid tool name (1-64 chars of [A-Za-z0-9_.-])".into(), + )) + } + }; + // A provider sees `a.b` as `a_b`, so two names that normalize alike + // would be one name on the wire — refused like an exact duplicate. + let wire = normalize_tool_name(&name); + if out.iter().any(|d| d.name == name || normalize_tool_name(&d.name) == wire) { + return Err(ebad(format!("duplicate tool name '{name}'"))); + } + if host.iter().any(|p| p.tool_name == name || normalize_tool_name(&p.tool_name) == wire) { + return Err(ebad(format!( + "'{name}' collides with a tool the plan pins — a dynamic tool needs a name of \ + its own" + ))); + } + let description = match e.get("description") { + None | Some(Value::Null) => String::new(), + Some(Value::String(s)) => s.clone(), + Some(other) => return Err(ebad(format!("`description` must be a string, not {other}"))), + }; + let input_schema = match e.get("input_schema") { + None | Some(Value::Null) => serde_json::json!({"type": "object", "properties": {}}), + Some(s) => { + // Annotations an MCP server publishes (`$schema`, `$id`, …) + // are accepted and dropped, never a refusal (#397); the + // pinned, offered and validating schema is the clean form. + let mut s = s.clone(); + strip_annotations(&mut s); + SchemaValidator::tool_schema() + .validate(&s) + .map_err(|err| ebad(format!("`input_schema` is not a usable schema: {err}")))?; + s + } + }; + let Some(Value::Object(d)) = e.get("dispatch") else { + return Err(ebad("`dispatch` must be an object naming a pinned Definition".into())); + }; + if let Some(k) = d.keys().find(|k| !DISPATCH_KEYS.contains(&k.as_str())) { + return Err(ebad(format!( + "unknown dispatch key {k:?} (accepted: {})", + DISPATCH_KEYS.join(", ") + ))); + } + let Some(target) = d.get("definition").and_then(Value::as_str) else { + return Err(ebad("`dispatch.definition` must name a pinned Definition".into())); + }; + let Some(def) = dispatchable + .iter() + .find(|p| p.tool_name == target) + .or_else(|| dispatchable.iter().find(|p| p.tool_hash.eq_ignore_ascii_case(target))) + .or_else(|| dispatchable.iter().find(|p| normalize_tool_name(&p.tool_name) == target)) + else { + return Err(RunError::UnresolvedRef { + what: format!( + "node '{node}' llm.tools_from {label} dispatches through '{target}', which \ + is neither in the node's `dispatch` list nor a host tool the plan pins \ + ({}) — a dynamic tool may only reach a Definition the plan's reviewer saw", + pinned_names(dispatchable) + ), + }); + }; + let locked = match d.get("locked") { + None | Some(Value::Null) => None, + Some(Value::Object(o)) => Some(Value::Object(o.clone())), + Some(other) => return Err(ebad(format!("`dispatch.locked` must be an object, not {other}"))), + }; + let args_into = match d.get("args_into") { + None | Some(Value::Null) => None, + Some(Value::String(s)) if !s.is_empty() => Some(s.clone()), + Some(other) => { + return Err(ebad(format!( + "`dispatch.args_into` must be a non-empty key, not {other}" + ))) + } + }; + // Locks are TOP-LEVEL keys of the Definition's input. A dotted key + // would be inserted literally (`"a.b"`, not `a` → `b`), and a lock on + // the `args_into` key would replace the model's arguments wholesale — + // the Definition's own `locked_params` included, which merge last. + let lock_keys = locked + .iter() + .chain(def.locked_params.iter()) + .filter_map(Value::as_object) + .flat_map(|o| o.keys()); + for k in lock_keys { + if k.contains('.') { + return Err(ebad(format!( + "lock key '{k}' is dotted — locks set top-level keys of the Definition's \ + input only" + ))); + } + if args_into.as_deref() == Some(k.as_str()) { + return Err(ebad(format!( + "lock key '{k}' is the `args_into` key, so it would replace the model's \ + arguments" + ))); + } + } + out.push(DynamicTool { + name, + description, + input_schema, + dispatch: DynamicDispatch { + definition: def.tool_name.clone(), + hash: def.tool_hash.clone(), + locked, + args_into, + // Filled from the store by the caller, which holds it. + definition_schema: None, + }, + }); + } + Ok(out) +} + +/// `over` merged over `base` at the top level, `over` winning — how a +/// Definition's own `locked_params` sit over a dynamic entry's `locked`. +pub(crate) fn merge_locks(base: Option<&Value>, over: Option<&Value>) -> Option { + match (base, over) { + (None, None) => None, + (Some(b), None) => Some(b.clone()), + (None, Some(o)) => Some(o.clone()), + (Some(Value::Object(b)), Some(Value::Object(o))) => { + let mut m = b.clone(); + for (k, v) in o { + m.insert(k.clone(), v.clone()); + } + Some(Value::Object(m)) + } + (_, Some(o)) => Some(o.clone()), + } +} + +/// Remove locked keys from a schema the model is OFFERED (#393): from +/// `properties` and from `required`. The pure rule lives in run-core, which +/// validates the model's arguments against the same stripped form. +pub(crate) fn strip_locked(schema: &mut Value, locked: &Value) { + areev_run_core::types::strip_locked_keys(schema, locked); +} + +/// OpenAI's strict structured output (`"strict": true`, which the runtime +/// sends) accepts a schema only when every object closes itself +/// (`additionalProperties: false`) and lists every property in `required`. +/// Checked at start, naming the first violation by JSON pointer, rather than +/// learnt from a 400 on the node's first turn. +fn strict_mode_violation(schema: &Value, at: &str) -> Option { + let Value::Object(o) = schema else { return None }; + // `type` may be a list (`["object", "null"]`): an object schema still. + let typed_object = match o.get("type") { + Some(Value::String(t)) => t == "object", + Some(Value::Array(ts)) => ts.iter().any(|t| t.as_str() == Some("object")), + _ => false, + }; + let is_object = typed_object || o.contains_key("properties"); + if is_object { + if o.get("additionalProperties") != Some(&Value::Bool(false)) { + return Some(format!("{} must set \"additionalProperties\": false", if at.is_empty() { "/" } else { at })); + } + let required: Vec<&str> = o + .get("required") + .and_then(Value::as_array) + .map(|r| r.iter().filter_map(Value::as_str).collect()) + .unwrap_or_default(); + if let Some(Value::Object(props)) = o.get("properties") { + for (k, sub) in props { + if !required.contains(&k.as_str()) { + return Some(format!( + "{}: property '{k}' must be listed in \"required\"", + if at.is_empty() { "/" } else { at } + )); + } + if let Some(v) = strict_mode_violation(sub, &format!("{at}/properties/{k}")) { + return Some(v); + } + } + } + } + if let Some(items) = o.get("items") { + if let Some(v) = strict_mode_violation(items, &format!("{at}/items")) { + return Some(v); + } + } + // A schema-valued `additionalProperties` is itself a schema strict mode + // walks (and on an object it is already a violation above). + if let Some(addl @ Value::Object(_)) = o.get("additionalProperties") { + if let Some(v) = strict_mode_violation(addl, &format!("{at}/additionalProperties")) { + return Some(v); + } + } + if let Some(Value::Array(alts)) = o.get("anyOf") { + for (n, alt) in alts.iter().enumerate() { + if let Some(v) = strict_mode_violation(alt, &format!("{at}/anyOf/{n}")) { + return Some(v); + } + } + } + None +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + #[test] + fn locked_keys_leave_the_offered_schema_and_its_required_list() { + let mut schema = json!({"type": "object", + "properties": {"url": {"type": "string"}, "body": {"type": "string"}}, + "required": ["url", "body"]}); + strip_locked(&mut schema, &json!({"url": "https://a.example/x"})); + assert_eq!( + schema, + json!({"type": "object", "properties": {"body": {"type": "string"}}, "required": ["body"]}) + ); + } + + #[test] + fn strict_mode_walks_nullable_objects_any_of_and_schema_valued_additional_properties() { + let ok = json!({"type": "object", "properties": {"a": {"type": "string"}}, + "required": ["a"], "additionalProperties": false}); + assert_eq!(strict_mode_violation(&ok, ""), None); + let nullable = json!({"type": ["object", "null"]}); + assert!(strict_mode_violation(&nullable, "").unwrap().contains("additionalProperties")); + let any_of = json!({"anyOf": [ok.clone(), {"type": "object", "properties": {"b": {"type": "string"}}, + "additionalProperties": false}]}); + assert!(strict_mode_violation(&any_of, "").unwrap().contains("/anyOf/1")); + let inner = json!({"type": "string", "additionalProperties": {"type": "object", + "properties": {"x": {"type": "string"}}, "required": ["x"]}}); + assert!(strict_mode_violation(&inner, "").unwrap().contains("/additionalProperties")); + } + + #[test] + fn a_definitions_own_lock_wins_over_a_dynamic_entrys() { + assert_eq!( + merge_locks(Some(&json!({"url": "a", "tool": "t"})), Some(&json!({"url": "b"}))), + Some(json!({"url": "b", "tool": "t"})) + ); + assert_eq!(merge_locks(None, None), None); + } +} diff --git a/crates/areev-run/src/manifest.rs b/crates/areev-run/src/manifest.rs index 4a1d4fef..3327af57 100644 --- a/crates/areev-run/src/manifest.rs +++ b/crates/areev-run/src/manifest.rs @@ -87,6 +87,22 @@ pub struct PinnedTool { /// pin, so existing manifests serialize byte-identically. #[serde(default, skip_serializing_if = "Option::is_none")] pub decide: Option, + /// The Definition's `locked_params` (#393): keys merged OVER a model's + /// arguments when an abstract node calls this tool, the lock winning, and + /// stripped from the schema the model is offered. Frozen here with the + /// rest of the resolution, so a run whose Definition locked nothing when + /// it started — every run before 1.13 — never starts locking mid-flight. + /// Absent for every Definition without locks, so existing manifests + /// serialize byte-identically. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub locked_params: Option, + /// An abstract node's frozen `llm` block (#391, #392): the plan's + /// declaration for this node, validated and normalized at start, with any + /// `tools_from` list resolved against the run input and pinned. Present + /// only on an abstract pin whose plan configured it, so existing manifests + /// serialize byte-identically. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub llm: Option, } /// The `executor` a decision node (C3) pins as. @@ -220,8 +236,23 @@ pub struct RunManifest { /// one: the scheduler then asks nothing, byte-for-byte as before. #[serde(default, skip_serializing_if = "Option::is_none")] pub decider: Option, + /// Definitions an abstract node's `llm.dispatch` pinned (#392): what its + /// dynamic tools dispatch through when no node binds it — resolved and + /// frozen like a bound Definition (code address, runtime, capabilities, + /// locks; `node` names the abstract node), checked at start like one, and + /// NEVER offered to a model as a tool of its own. Kept apart from + /// `pinned`, which is one entry per plan node. Empty — and absent — on + /// every manifest that declares none, so stored manifests keep their bytes. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub dispatch_pins: Vec, + /// Start-time warnings (`RUN-Wnnn …`), frozen with the run so `run + /// inspect` can say what the start accepted but did not act on. Empty — + /// and absent — on every run that raised none. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub warnings: Vec, } + /// The model configuration a run is pinned to (#287). /// /// `tag` is an opaque host string — a configuration hash, typically — so a @@ -407,6 +438,7 @@ impl RunManifest { // here, at start — what the run reads, whatever the row says later. crate::memread::pin_queries(&mut reads, m)?; let mut pinned = Vec::with_capacity(plan.nodes.len()); + let mut warnings: Vec = Vec::new(); for (i, node) in plan.nodes.iter().enumerate() { if let Some(spec) = reads.remove(node) { let op = spec.get("op").and_then(|o| o.as_str()).unwrap_or_default(); @@ -422,6 +454,8 @@ impl RunManifest { read: Some(spec), ask_kind: None, decide: None, + locked_params: None, + llm: None, }); continue; } @@ -447,9 +481,13 @@ impl RunManifest { read: None, ask_kind: None, decide: None, + locked_params: None, + llm: None, } } else { - pin_from_definition(node, &h, &g)? + let pin = pin_from_definition(node, &h, &g)?; + warnings.extend(locked_params_warning(&pin, &g)); + pin } } None => { @@ -458,7 +496,11 @@ impl RunManifest { // instruction the model interprets — which needs the // tool-calling seam configured. match find_definition_by_name(m, ns, node) { - Some((h, g)) => pin_from_definition(node, &h, &g)?, + Some((h, g)) => { + let pin = pin_from_definition(node, &h, &g)?; + warnings.extend(locked_params_warning(&pin, &g)); + pin + } // The misroute FIRST, before either fallback (#230). // A node whose name matches a real Definition sitting // in the plan's own namespace is a run started in the @@ -493,6 +535,8 @@ impl RunManifest { read: None, ask_kind: None, decide: None, + locked_params: None, + llm: None, }, None => return Err(RunError::NoToolLlm { node: node.clone() }), } @@ -500,6 +544,104 @@ impl RunManifest { }; pinned.push(resolved); } + // The per-node `llm` blocks (#391, #392), validated against the pins + // just resolved — a `tools` subset or a dynamic tool's dispatch may + // only name what the plan pinned — and FROZEN onto each abstract pin. + // `tools_from` resolves against the run input here, once. + // A `dispatch` entry resolves exactly as an unbound node's same-named + // Definition does — the run's namespace, then the plan grain's — or by + // content address, and pins like a binding. + let mut resolve_dispatch = |node: &str, entry: &str| -> std::result::Result { + let by_hash = entry.len() == 64 && entry.chars().all(|c| c.is_ascii_hexdigit()); + let found = if by_hash { + Hash::from_hex(entry).ok().and_then(|h| m.get_stored(&h).ok().map(|g| (h, g))) + } else { + find_definition_by_name(m, ns, entry).or_else(|| match plan_ns.as_deref() { + Some(p) if p != ns => find_definition_by_name(m, p, entry), + _ => None, + }) + }; + let Some((h, g)) = found else { + return Err(RunError::UnresolvedRef { + what: format!( + "node '{node}' llm.dispatch names '{entry}', which resolves to no Tool \ + Definition — not in namespace '{ns}', where this run reads, nor in the \ + plan's own namespace{}", + plan_ns.as_deref().map(|p| format!(" '{p}'")).unwrap_or_default() + ), + }); + }; + let mut pin = pin_from_definition(node, &h, &g)?; + if pin.executor != "host" { + return Err(RunError::InvalidPlan { + why: format!( + "node '{node}' llm.dispatch names '{entry}', a {} tool — a dynamic tool \ + dispatches through a host tool only", + pin.executor + ), + }); + } + pin.node = node.to_string(); + Ok(pin) + }; + let (blocks, dispatch_pins) = crate::llmcfg::parse_llm( + fields.and_then(|f| f.get(crate::llmcfg::LLM_FIELD)), + plan, + &pinned, + &input, + &mut resolve_dispatch, + )?; + // A dispatch pin may not be ANOTHER version of a Definition a node + // binds under the same name: the strict-argument table is keyed by + // `tool_name`, and the second would overwrite the first's schema. + // Nor may two dispatch pins be different versions of one name, in one + // node or across nodes: the same table, the same overwrite. + for (n, d) in dispatch_pins.iter().enumerate() { + if let Some(o) = dispatch_pins[..n] + .iter() + .find(|o| o.tool_name == d.tool_name && o.tool_hash != d.tool_hash) + { + return Err(RunError::InvalidPlan { + why: format!( + "llm.dispatch pins two Definitions named {}: {} (node '{}') and {} \ + (node '{}') — one name, one version per run", + d.tool_name, o.tool_hash, o.node, d.tool_hash, d.node + ), + }); + } + } + for d in &dispatch_pins { + if let Some(b) = pinned + .iter() + .find(|p| p.executor == "host" && p.tool_name == d.tool_name && p.tool_hash != d.tool_hash) + { + return Err(RunError::InvalidPlan { + why: format!( + "node '{}' llm.dispatch pins {} ({}), but node '{}' binds a different \ + Definition of that name ({}) — dispatch through the bound one, or give \ + the other version its own tool_name", + d.node, d.tool_name, d.tool_hash, b.node, b.tool_hash + ), + }); + } + } + for (node, mut block) in blocks { + // The dispatch Definition's own schema, frozen beside the entry: + // the merged input is validated against it on every call. + for d in &mut block.dynamic_tools { + let h = Hash::from_hex(&d.dispatch.hash).map_err(|_| RunError::UnresolvedRef { + what: format!("dispatch hash {} is not a content address", d.dispatch.hash), + })?; + d.dispatch.definition_schema = m + .get_stored(&h) + .ok() + .and_then(|g| g.to_tool().ok()) + .and_then(|t| t.input_schema); + } + if let Some(p) = pinned.iter_mut().find(|p| p.node == node) { + p.llm = Some(block); + } + } // Reducer table (§6.5): declared on the Workflow grain, validated // here — an unknown name fails at run start, not at first merge. let mut reducers: BTreeMap = BTreeMap::new(); @@ -546,6 +688,8 @@ impl RunManifest { harness_ns: None, allow_confirmation_asks: false, decider: None, + dispatch_pins, + warnings, }) } @@ -653,6 +797,22 @@ impl RunManifest { .unwrap_or(areev_run_core::DEFAULT_MAX_EFFECTS_PER_ATTEMPT) } + /// The host Definition pinned at `hash` — a node's binding or an + /// `llm.dispatch` pin (#392). How a model-issued call finds what it + /// executes as. + pub fn host_pin_by_hash(&self, hash: &str) -> Option<&PinnedTool> { + self.pinned + .iter() + .chain(&self.dispatch_pins) + .find(|p| p.executor == "host" && p.tool_hash == hash) + } + + /// Every pin that names code this run may execute: the nodes' and the + /// `llm.dispatch` pins'. What start checks against the host's pins. + pub fn code_pins(&self) -> impl Iterator { + self.pinned.iter().chain(&self.dispatch_pins) + } + /// The executor vector the scheduler env consumes, in node order. An /// abstract node's offered tools are the manifest's HOST-pinned tools — /// derived, not stored, so the manifest stays minimal and the offer is @@ -670,11 +830,35 @@ impl RunManifest { .iter() .filter(|p| p.executor == "host") .filter(|p| seen.insert(p.tool_name.clone())) - .map(|p| areev_run_core::OfferedTool { - tool_name: p.tool_name.clone(), - tool_hash: p.tool_hash.clone(), + .map(|p| { + let mut t = areev_run_core::OfferedTool::new(p.tool_name.clone(), p.tool_hash.clone()); + t.locked = p.locked_params.clone(); + t }) .collect(); + // A node's own offer (#391, #392): its `tools` subset of the pinned + // set — manifest order, narrowing only — then the dynamic tools its + // block pinned from the run input, each locked by its entry and, over + // that, by the Definition it dispatches. + let offer_for = |p: &PinnedTool| -> Vec { + let Some(cfg) = &p.llm else { return offered.clone() }; + let mut tools: Vec = match &cfg.tools { + Some(names) => offered.iter().filter(|t| names.contains(&t.tool_name)).cloned().collect(), + None => offered.clone(), + }; + for d in &cfg.dynamic_tools { + let def_lock = self + .host_pin_by_hash(&d.dispatch.hash) + .and_then(|q| q.locked_params.as_ref()); + tools.push(areev_run_core::OfferedTool { + tool_name: d.name.clone(), + tool_hash: d.dispatch.hash.clone(), + locked: crate::llmcfg::merge_locks(d.dispatch.locked.as_ref(), def_lock), + dynamic: Some(Box::new(d.clone())), + }); + } + tools + }; self.pinned .iter() .map(|p| match p.executor.as_str() { @@ -695,7 +879,10 @@ impl RunManifest { spec, } } - "abstract" => NodeExecutor::Abstract { tools: offered.clone() }, + "abstract" => NodeExecutor::Abstract { + tools: offer_for(p), + llm: p.llm.clone().map(Box::new), + }, // Never Host, for the memory-read reason: a decision that // fell through to `--tool-cmd` would let a tool answer it. DECIDE_EXECUTOR => NodeExecutor::Decide { @@ -1053,6 +1240,24 @@ pub fn pin_from_definition( } }; let decide = if decide_node { Some(parse_decide(node, g.fields.get("decide"))?) } else { None }; + // `locked_params` (#393): an object, or refused — the write path checks + // it, but a grain can arrive by sync or in a pack from an implementation + // we do not vouch for, and a lock that silently failed to apply is a + // parameter the model was trusted with by mistake. + let locked_params = match g.fields.get("locked_params") { + None | Some(serde_json::Value::Null) => None, + Some(serde_json::Value::Object(o)) if o.is_empty() => None, + Some(v @ serde_json::Value::Object(_)) if executor == "host" && !decide_node => Some(v.clone()), + // A client or decision tool: nothing a model calls, so nothing to + // lock. Ignored, as 1.12 ignored it; the start records RUN-W001 + // (`locked_params_warning`). + Some(serde_json::Value::Object(_)) => None, + Some(other) => { + return Err(RunError::InvalidPlan { + why: format!("node '{node}' binds a Definition whose locked_params is not an object: {other}"), + }) + } + }; Ok(PinnedTool { node: node.to_string(), tool_hash: h.to_hex(), @@ -1065,6 +1270,31 @@ pub fn pin_from_definition( read: None, ask_kind, decide, + locked_params, + llm: None, + }) +} + +/// The `RUN-W001` text for a pin whose Definition declares `locked_params` +/// the runtime ignores (a client or decision tool), or `None`. No model calls +/// such a tool, so the locks have nothing to apply to; 1.12 accepted them +/// silently, and refusing them now would break a plan that ran yesterday, so +/// they are ignored and the start says so. +pub fn locked_params_warning( + pin: &PinnedTool, + g: &areev_core::format::deserialize::DeserializedGrain, +) -> Option { + let declared = g + .fields + .get("locked_params") + .and_then(|v| v.as_object()) + .is_some_and(|o| !o.is_empty()); + (declared && pin.locked_params.is_none() && pin.executor != "host").then(|| { + format!( + "RUN-W001: node '{}' binds {} ({}), a {} tool, which declares locked_params — no \ + model calls it, so they are ignored", + pin.node, pin.tool_name, pin.tool_hash, pin.executor + ) }) } @@ -1245,6 +1475,8 @@ mod tests { harness_ns: None, allow_confirmation_asks: false, decider: None, + dispatch_pins: Vec::new(), + warnings: Vec::new(), } } @@ -1261,6 +1493,8 @@ mod tests { read: None, ask_kind: None, decide: None, + locked_params: None, + llm: None, } } @@ -1284,6 +1518,8 @@ mod tests { read: None, ask_kind: None, decide: None, + locked_params: None, + llm: None, }, host_pin("reply_done", "reply_email"), host_pin("reply_rejected", "reply_email"), @@ -1291,7 +1527,7 @@ mod tests { let executors = m.executors(); assert_eq!(executors.len(), 4); match &executors[1] { - NodeExecutor::Abstract { tools } => { + NodeExecutor::Abstract { tools, .. } => { let names: Vec<&str> = tools.iter().map(|t| t.tool_name.as_str()).collect(); assert_eq!(names, vec!["parse_attachments", "reply_email"]); assert_eq!(tools[1].tool_hash, "hash-reply_email"); @@ -1329,6 +1565,8 @@ mod tests { read, ask_kind: None, decide: None, + locked_params: None, + llm: None, }; let mut m = bare(); m.pinned = vec![ @@ -1338,7 +1576,7 @@ mod tests { ]; let executors = m.executors(); assert_eq!(executors[1], NodeExecutor::MemoryRead { op: "entity_at".into(), spec }); - let NodeExecutor::Abstract { tools } = &executors[2] else { panic!("{executors:?}") }; + let NodeExecutor::Abstract { tools, .. } = &executors[2] else { panic!("{executors:?}") }; let offered: Vec<&str> = tools.iter().map(|t| t.tool_name.as_str()).collect(); assert_eq!(offered, vec!["parse_attachments"], "a read is never offered to a model"); @@ -1391,4 +1629,55 @@ mod tests { let json = serde_json::to_string(&m).unwrap(); assert!(json.contains(r#""max_effects_per_attempt":40"#), "{json}"); } + /// #391/#393: a pin without a lock or a block serializes as it always did, + /// so every stored manifest keeps its address. + #[test] + fn a_pin_without_a_lock_or_a_block_keeps_its_bytes() { + let json = serde_json::to_string(&host_pin("parse", "parse")).unwrap(); + assert!(!json.contains("locked_params") && !json.contains("\"llm\""), "{json}"); + } + + /// A node's offer is its `tools` subset in manifest order, then its pinned + /// dynamic tools — each locked by its entry and, over that, by the + /// Definition it dispatches; every other abstract node keeps the full set. + #[test] + fn a_nodes_offer_is_its_subset_then_its_dynamic_tools() { + let mut m = bare(); + let mut mcp = host_pin("mcp", "mcp.call"); + mcp.locked_params = Some(json!({"credential": "gateway"})); + let cfg = areev_run_core::LlmNodeConfig { + tools: Some(vec!["lookup".into()]), + dynamic_tools: vec![areev_run_core::DynamicTool { + name: "paid_to".into(), + description: String::new(), + input_schema: json!({"type": "object"}), + dispatch: areev_run_core::DynamicDispatch { + definition: "mcp.call".into(), + hash: "hash-mcp.call".into(), + locked: Some(json!({"url": "u", "credential": "loses"})), + args_into: Some("arguments".into()), + definition_schema: None, + }, + }], + ..Default::default() + }; + m.pinned = vec![ + host_pin("lookup", "lookup"), + mcp, + PinnedTool { executor: "abstract".into(), llm: Some(cfg), ..host_pin("agent", "agent") }, + PinnedTool { executor: "abstract".into(), ..host_pin("other", "other") }, + ]; + let ex = m.executors(); + let NodeExecutor::Abstract { tools, llm } = &ex[2] else { panic!("{ex:?}") }; + let names: Vec<&str> = tools.iter().map(|t| t.tool_name.as_str()).collect(); + assert_eq!(names, vec!["lookup", "paid_to"]); + assert_eq!(tools[1].tool_hash, "hash-mcp.call"); + assert_eq!(tools[1].locked, Some(json!({"url": "u", "credential": "gateway"}))); + assert!(llm.is_some()); + let NodeExecutor::Abstract { tools, llm } = &ex[3] else { panic!("{ex:?}") }; + let names: Vec<&str> = tools.iter().map(|t| t.tool_name.as_str()).collect(); + assert_eq!(names, vec!["lookup", "mcp.call"]); + assert_eq!(tools[1].locked, Some(json!({"credential": "gateway"}))); + assert!(llm.is_none()); + } } diff --git a/crates/areev-run/src/runner.rs b/crates/areev-run/src/runner.rs index 95945679..c16d8ca0 100644 --- a/crates/areev-run/src/runner.rs +++ b/crates/areev-run/src/runner.rs @@ -482,6 +482,40 @@ impl Runner { } } +/// The request fields of one LLM turn, read off its JOURNALED input (#391): +/// the node's `llm` block as the scheduler wrote it under `llm`, else the +/// run's defaults. Read from the journal rather than the manifest so the +/// request a turn went out with is a function of the record alone. +pub(crate) struct TurnRequest { + pub system: Option, + /// `None` = send no temperature (`"omit"`). + pub temperature: Option, + pub max_tokens: u32, + pub response_format: Option, + pub parallel_tool_calls: Option, +} + +impl TurnRequest { + pub(crate) fn from_input(input: &Value, manifest: &RunManifest) -> Self { + let block = input.get("llm"); + let field = |k: &str| block.and_then(|b| b.get(k)); + TurnRequest { + system: field("system").and_then(Value::as_str).map(str::to_string), + temperature: match field("temperature") { + Some(Value::String(s)) if s == "omit" => None, + Some(t) => t.as_f64().or(Some(LLM_TEMPERATURE)), + None => Some(LLM_TEMPERATURE), + }, + max_tokens: field("max_tokens") + .and_then(Value::as_u64) + .and_then(|n| u32::try_from(n).ok()) + .unwrap_or_else(|| manifest.llm_max_tokens.unwrap_or(DEFAULT_LLM_MAX_TOKENS)), + response_format: field("response_format").and_then(areev_llm::ResponseFormat::from_openai), + parallel_tool_calls: field("parallel_tool_calls").and_then(Value::as_bool), + } + } +} + /// Journal-shaped transcript messages → the tool-calling seam's shape. /// Pure translation, on the driver thread; the pool never parses state. fn seam_messages(input: &Value) -> Vec { @@ -666,6 +700,87 @@ impl Runner { PlanGraph::build(&wf) } + /// The Definition one offered tool is shown to the model as. A pinned + /// Definition is read by its frozen address, with its locked keys taken + /// out of the schema (#393) — the model is not invited to set what it + /// cannot choose. A dynamic tool (#392) is the entry the manifest pinned: + /// its own name, description and schema, never the dispatch Definition's. + fn offered_definition( + &self, + t: &areev_run_core::OfferedTool, + ) -> Result { + if let Some(d) = &t.dynamic { + let mut def = areev_core::types::Tool::new(&d.name) + .kind(areev_core::types::ToolKind::Definition); + if !d.description.is_empty() { + def = def.tool_description(&d.description); + } + // The same stripped form the scheduler validates the model's + // arguments against (run-core `DynamicTool::model_schema`). + def.input_schema = Some(d.model_schema(t.locked.as_ref())); + return Ok(def); + } + let h = Hash::from_hex(&t.tool_hash).map_err(err_run)?; + let mut def = self + .facade + .with_store(|m| m.get(&h)) + .map_err(err_run)? + .to_tool() + .map_err(err_run)?; + if let (Some(lock), Some(schema)) = (&t.locked, def.input_schema.as_mut()) { + crate::llmcfg::strip_locked(schema, lock); + } + Ok(def) + } + + /// Pseudonymize a dynamic tool's prose for the model: its description, + /// and every `description`, `title`, `default` and `examples` value in its + /// schema. A no-op without an `egress` boundary. + fn pseudonymize_tool_text(&self, def: &mut areev_core::types::Tool) -> Result<(), RunError> { + if !self.anon_boundary_active()? { + return Ok(()); + } + const PROSE: [&str; 4] = ["description", "title", "default", "examples"]; + // One walk order for both passes, so the values come back to the + // slots they left. + fn walk(schema: &mut Value, f: &mut dyn FnMut(&mut Value)) { + let Value::Object(o) = schema else { return }; + for (k, v) in o.iter_mut() { + match k.as_str() { + k if PROSE.contains(&k) => f(v), + "properties" => { + if let Value::Object(props) = v { + for p in props.values_mut() { + walk(p, f); + } + } + } + "items" | "additionalProperties" => walk(v, f), + _ => {} + } + } + } + let mut schema = def.input_schema.clone().unwrap_or(Value::Null); + let mut slots = vec![Value::String(def.tool_description.clone().unwrap_or_default())]; + walk(&mut schema, &mut |v| slots.push(v.clone())); + let masked = self.pseudonymize_for_model(&Value::Array(slots))?; + let mut masked = masked.as_array().cloned().unwrap_or_default().into_iter(); + if let Some(Value::String(d)) = masked.next() { + if def.tool_description.is_some() { + def.tool_description = Some(d); + } + } + walk(&mut schema, &mut |v| { + if let Some(m) = masked.next() { + *v = m; + } + }); + if def.input_schema.is_some() { + def.input_schema = Some(schema); + } + Ok(()) + } + fn load_manifest(&self, run_id: &str) -> Result { self.facade.with_store(|m| RunManifest::load(m, run_id)) } @@ -754,7 +869,7 @@ impl Runner { // otherwise take a lease, write a manifest, and fail on first // dispatch, leaving a terminal run to explain. The address is named // so pinning it is a copy-paste. - for p in &manifest.pinned { + for p in manifest.code_pins() { if let Some(uri) = &p.executor_uri { if !self.executor.code_allowed(&p.tool_hash, uri) { return Err(RunError::CodeExecRefused { @@ -2270,10 +2385,19 @@ impl Runner { provider: in_agent .then(|| self.llm.as_ref().map(|l| l.provider().to_string())) .flatten(), - max_tokens: is_llm.then(|| { - manifest.llm_max_tokens.unwrap_or(DEFAULT_LLM_MAX_TOKENS) - }), - temperature: is_llm.then_some(LLM_TEMPERATURE), + // What THIS request carries (#391): the node's + // own ceiling and temperature when its block sets + // them, nothing for an omitted temperature — a span + // must not claim a parameter that never went out. + max_tokens: is_llm + .then(|| TurnRequest::from_input(&input, manifest).max_tokens), + temperature: if is_llm { + TurnRequest::from_input(&input, manifest).temperature.and_then( + |t| self.llm.as_ref().map_or(Some(t), |l| l.effective_temperature(t)), + ) + } else { + None + }, }); intents.insert(key, h); intents_written += 1; @@ -2374,7 +2498,7 @@ impl Runner { // (transcript translation + pinned Definitions from // the store), then hand the pure job to the pool. let llm_prepared = if key.kind == areev_run_core::EffectKind::Llm { - let areev_run_core::NodeExecutor::Abstract { tools } = &executor + let areev_run_core::NodeExecutor::Abstract { tools, .. } = &executor else { return Err(RunError::Storage { detail: "llm effect on a non-abstract node".into(), @@ -2414,13 +2538,18 @@ impl Runner { }; let mut defs = Vec::with_capacity(offered.len()); for t in offered { - let h = Hash::from_hex(&t.tool_hash).map_err(err_run)?; - let def = self - .facade - .with_store(|m| m.get(&h)) - .map_err(err_run)? - .to_tool() - .map_err(err_run)?; + let mut def = self.offered_definition(t)?; + // A dynamic tool's text came from the run + // INPUT — host data, possibly naming a + // person — so its description and the prose + // annotations of its schema cross the + // boundary pseudonymized, as the input does. + // Structural strings (types, enums, patterns) + // are left alone: the arguments are validated + // against them as stored. + if t.dynamic.is_some() { + self.pseudonymize_tool_text(&mut def)?; + } defs.push(def); } // §6.10 TokenChunk sink: only when observed — @@ -2440,13 +2569,32 @@ impl Runner { }) as crate::executor::TokenSink }); let to_model = self.pseudonymize_for_model(&input)?; + // The node's request fields (#391), read off the + // turn's own journaled `llm` — what the scheduler + // wrote is what is sent, and what verify sees. + // Read off `to_model`, never the raw input: the + // node's system text crosses the model boundary + // exactly as its messages do, so an `egress` + // policy pseudonymizes it too (#391 review). + // Only the PROSE is taken from the masked copy: + // the response schema goes out as declared, since + // the answer is checked against it as declared. + let mut req = TurnRequest::from_input(&input, manifest); + req.system = TurnRequest::from_input(&to_model, manifest).system; Some(crate::executor::PreparedLlm { messages: seam_messages(&to_model), tools: defs, - max_tokens: manifest - .llm_max_tokens - .unwrap_or(DEFAULT_LLM_MAX_TOKENS), + max_tokens: req.max_tokens, on_token, + system: req.system, + temperature: req.temperature, + response_format: req.response_format, + parallel_tool_calls: req.parallel_tool_calls, + record_sent: input.get("llm").is_some() + || self + .llm + .as_ref() + .is_some_and(|l| l.request_profile_digest().is_some()), }) } else { None @@ -2457,12 +2605,37 @@ impl Runner { // touches no store handle. `get_blob` verifies the // digest on every read, so the bytes handed to the // pool always are the address the manifest pinned. - let code_prepared = match manifest - .pinned - .iter() - .find(|p| p.node == key.node) - .filter(|p| p.executor_uri.is_some()) - { + // A model-issued call (inside an abstract node) + // reaches the Definition it was offered as, found by + // its frozen address — a lookup by NODE finds the + // abstract node's own pin, which names no code, and + // so used to send a call to a code-carrying + // Definition to `--tool-cmd` instead of its blob. A + // dynamic tool (#392) is journaled under its own name + // but EXECUTES as that Definition: its name is what + // the host executor and the broker's grants know. + let flow_target = match &executor { + areev_run_core::NodeExecutor::Host { tool_hash, tool_name } + if key.kind == areev_run_core::EffectKind::Tool + && manifest + .pinned + .iter() + .any(|p| p.node == key.node && p.executor == "abstract") => + { + manifest + .host_pin_by_hash(tool_hash) + .map(|p| (p, p.tool_name != *tool_name)) + } + _ => None, + }; + let exec_name = flow_target + .filter(|(_, renamed)| *renamed) + .map(|(p, _)| p.tool_name.clone()); + let code_pin = match flow_target { + Some((p, _)) => Some(p), + None => manifest.pinned.iter().find(|p| p.node == key.node), + }; + let code_prepared = match code_pin.filter(|p| p.executor_uri.is_some()) { Some(pin) => { let uri = pin.executor_uri.as_deref().unwrap(); let bytes = self @@ -2523,6 +2696,7 @@ impl Runner { idempotency_key: idem, llm: llm_prepared, code: code_prepared, + exec_name, }); in_flight += 1; } @@ -3181,6 +3355,8 @@ impl Runner { pinned: manifest .pinned .iter() + // `llm.dispatch` pins (#392) follow the nodes' rows, marked. + .chain(&manifest.dispatch_pins) .map(|p| { // The resolution, not a summary of it (#230). A node // bound to a Definition that names a code blob used to @@ -3205,6 +3381,21 @@ impl Runner { if let Some(read) = &p.read { row_obj.insert("read".into(), read.clone()); } + if let Some(lock) = &p.locked_params { + row_obj.insert("locked_params".into(), lock.clone()); + } + if manifest.dispatch_pins.iter().any(|d| std::ptr::eq(d, p)) { + row_obj.insert("dispatch_only".into(), json!(true)); + } + // The node's frozen `llm` block (#391), dynamic tools + // as pinned (#392) — what a resume will send, not what + // the plan says today. + if let Some(llm) = &p.llm { + row_obj.insert( + "llm".into(), + serde_json::to_value(llm).unwrap_or(Value::Null), + ); + } row }) .collect(), @@ -3219,6 +3410,7 @@ impl Runner { phase, spent: scheduler.get("spent").cloned(), pending_asks: asks, + warnings: manifest.warnings.clone(), limits: json!({ "llm_max_tokens": manifest.llm_reserve_tokens(), "max_effects_per_attempt": manifest.max_effects_per_attempt(), diff --git a/crates/areev-run/src/shadow.rs b/crates/areev-run/src/shadow.rs index a25c36bd..6023409f 100644 --- a/crates/areev-run/src/shadow.rs +++ b/crates/areev-run/src/shadow.rs @@ -774,6 +774,15 @@ impl ShadowCtx<'_> { true, ), })?; + // Locks come from what the INCUMBENT froze, never from re-reading + // the Definition: a run started before locks were applied (any 1.12 + // run) froze none, and rehearsing it with the Definition's locks + // would replay a different validation and diverge on an unchanged + // plan. A Definition the incumbent never pinned locks nothing here. + for p in manifest.pinned.iter_mut().chain(manifest.dispatch_pins.iter_mut()) { + p.locked_params = + incumbent.host_pin_by_hash(&p.tool_hash).and_then(|q| q.locked_params.clone()); + } manifest.llm_max_tokens = incumbent.llm_max_tokens; manifest.max_effects_per_attempt = incumbent.max_effects_per_attempt; manifest.llm_tool_result_chars = incumbent.llm_tool_result_chars; @@ -1072,7 +1081,9 @@ pub(crate) fn arg_schemas_for( manifest: &RunManifest, ) -> Result, RunError> { let mut schemas = std::collections::BTreeMap::new(); - for p in &manifest.pinned { + // `llm.dispatch` pins (#392) included: a dynamic tool's merged input is + // judged under the Definition it dispatches, strict like any other. + for p in manifest.code_pins() { // A decision node's strict schema applies to the `{state, questions}` // the driver builds for it (C3), as a strict host tool's applies to // a model's call. diff --git a/crates/areev-run/tests/anon_boundary_tests.rs b/crates/areev-run/tests/anon_boundary_tests.rs index 166863ef..94b4d1f4 100644 --- a/crates/areev-run/tests/anon_boundary_tests.rs +++ b/crates/areev-run/tests/anon_boundary_tests.rs @@ -456,3 +456,56 @@ fn a_model_tool_call_with_an_invented_placeholder_is_refused() { let RunSession::Finished { outcome, .. } = session else { panic!("expected finish") }; assert_eq!(outcome, RunOutcome::Completed, "{outcome:?}"); } + +/// #391 review: a node's own `system` text and a dynamic tool's description +/// and schema prose cross the model boundary pseudonymized, like the prompt. +#[test] +fn a_nodes_system_text_and_dynamic_tool_prose_are_pseudonymized() { + struct Seeing { + inner: Arc, + sent: Mutex>, + } + impl ToolCallLlm for Seeing { + fn model(&self) -> &str { + "scripted" + } + fn call(&self, req: &ToolCallRequest<'_>) -> Result { + let tools: Vec = req + .tools + .iter() + .map(|t| format!("{:?} {:?}", t.tool_description, t.input_schema)) + .collect(); + self.sent.lock().unwrap().push(format!("{:?} {}", req.system, tools.join(" "))); + self.inner.call(req) + } + } + let rig = Rig::new(Some(r#"{"mode": "egress", "scope": "memory"}"#)); + let post = Tool::new("post").kind(ToolKind::Definition).created_at(500).namespace(NS); + let dh = rig.facade.with_store(|m| m.add(&post)).unwrap(); + let mut wf = Workflow::new(vec!["extract".into(), "post".into()]) + .edge("extract", "post") + .bind("post", &dh.to_hex()) + .created_at(800) + .namespace(NS); + wf.common.extra_fields.insert( + "llm".into(), + json!({"extract": {"system": format!("Escalate to {SUPPLIER}."), "tools_from": "/tools"}}), + ); + let plan = rig.facade.with_store(|m| m.add(&wf)).unwrap(); + let llm = Arc::new(Seeing { inner: ScriptedLlm::new(vec![done("{}")]), sent: Mutex::new(Vec::new()) }); + let input = json!({"tools": [{ + "name": "notify", + "description": format!("mails {SUPPLIER}"), + "input_schema": {"type": "object", "properties": {"to": {"type": "string", + "description": format!("e.g. {SUPPLIER}")}}}, + "dispatch": {"definition": "post"}, + }]}); + let session = rig + .runner(Arc::clone(&llm) as Arc) + .start(&plan, "anon-sys", input, &opts()) + .unwrap(); + assert!(matches!(session, RunSession::Finished { outcome: RunOutcome::Completed, .. })); + let sent = llm.sent.lock().unwrap().join("\n"); + assert!(!sent.contains(SUPPLIER), "the address reached the model: {sent}"); + assert!(sent.contains("[EMAIL_"), "it was replaced by a token: {sent}"); +} diff --git a/crates/areev-run/tests/llm_node_tests.rs b/crates/areev-run/tests/llm_node_tests.rs new file mode 100644 index 00000000..9d49cbda --- /dev/null +++ b/crates/areev-run/tests/llm_node_tests.rs @@ -0,0 +1,1361 @@ +//! The per-node `llm` block (#391), dynamic tools declared in the run input +//! (`tools_from`, #392) and `locked_params` (#393), end to end over a real +//! store: what the plan declares is frozen into the manifest, sent to the +//! provider, journaled as sent, and replayed by `verify` from the journal +//! alone — the model is never called again. + +use areev_cal::AreevFacade; +use areev_core::error::Hash; +use areev_core::types::{Grain, Tool, ToolKind, Workflow}; +use areev_llm::{ + ChatMessage, ResponseFormat, StopReason, ToolCallError, ToolCallLlm, ToolCallOut, + ToolCallRequest, ToolCallResponse, Usage, +}; +use areev_run::{ + ExecResult, HostToolExecutor, PreparedCode, RunOptions, RunSession, Runner, ScriptedClock, +}; +use areev_run_core::{EffectKind, RunOutcome}; +use areev_store::Areev; +use serde_json::{json, Value}; +use std::collections::VecDeque; +use std::sync::{Arc, Mutex}; +use tempfile::TempDir; + +// ---- rig ------------------------------------------------------------------- + +/// Every host execution, as the executor saw it: (name, input, ran as code). +#[derive(Default)] +struct TestExec { + seen: Mutex)>>, + allow_code: bool, +} + +impl TestExec { + fn seen(&self) -> Vec<(String, Value, Option)> { + self.seen.lock().unwrap().clone() + } +} + +impl HostToolExecutor for TestExec { + fn execute(&self, tool_name: &str, _hash: &str, input: &Value, _idem: &str) -> ExecResult { + self.seen.lock().unwrap().push((tool_name.to_string(), input.clone(), None)); + ExecResult::Ok(json!({ "ran": tool_name, "rows": [1, 2, 3] })) + } + fn code_allowed(&self, _tool_hash: &str, _uri: &str) -> bool { + self.allow_code + } + fn execute_code( + &self, + tool_name: &str, + _tool_hash: &str, + code: &PreparedCode, + input: &Value, + _idem: &str, + ) -> ExecResult { + self.seen + .lock() + .unwrap() + .push((tool_name.to_string(), input.clone(), Some(code.uri.clone()))); + ExecResult::Ok(json!({ "status": 200, "result": {"payee": "ACME"} })) + } +} + +/// One request as the provider received it. +#[derive(Debug, Clone)] +struct Sent { + system: Option, + temperature: f64, + omit_temperature: bool, + max_tokens: u32, + response_format: Option, + parallel_tool_calls: Option, + tools: Vec<(String, Option)>, + first_user: Value, + last_user: String, +} + +struct CapturingLlm { + script: Mutex>, + sent: Mutex>, +} + +impl CapturingLlm { + fn new(script: Vec) -> Arc { + Arc::new(CapturingLlm { script: Mutex::new(script.into()), sent: Mutex::new(Vec::new()) }) + } + fn sent(&self) -> Vec { + self.sent.lock().unwrap().clone() + } +} + +impl ToolCallLlm for CapturingLlm { + fn model(&self) -> &str { + "gpt-5-mini" + } + fn provider(&self) -> &'static str { + "openai" + } + fn call(&self, req: &ToolCallRequest<'_>) -> Result { + let users: Vec<&String> = req + .messages + .iter() + .filter_map(|m| match m { + ChatMessage::User(s) => Some(s), + _ => None, + }) + .collect(); + self.sent.lock().unwrap().push(Sent { + system: req.system.clone(), + temperature: req.temperature, + omit_temperature: req.omit_temperature, + max_tokens: req.max_tokens, + response_format: req.response_format.clone(), + parallel_tool_calls: req.parallel_tool_calls, + tools: req.tools.iter().map(|t| (t.tool_name.clone(), t.input_schema.clone())).collect(), + first_user: users + .first() + .and_then(|s| serde_json::from_str(s).ok()) + .unwrap_or(Value::Null), + last_user: users.last().map(|s| s.to_string()).unwrap_or_default(), + }); + self.script.lock().unwrap().pop_front().ok_or(ToolCallError { + retryable: false, + message: "script exhausted".into(), + ..Default::default() + }) + } +} + +fn calls(calls: Vec<(&str, &str, Value)>) -> ToolCallResponse { + ToolCallResponse::new( + None, + calls + .into_iter() + .map(|(id, name, arguments)| ToolCallOut { + id: id.into(), + name: name.into(), + arguments, + arguments_raw: None, + }) + .collect(), + StopReason::ToolUse, + Usage { input_tokens: 10, output_tokens: 5, cache_read_tokens: None }, + ) +} + +fn answer(text: &str) -> ToolCallResponse { + ToolCallResponse::new( + Some(text.into()), + vec![], + StopReason::EndTurn, + Usage { input_tokens: 20, output_tokens: 7, cache_read_tokens: None }, + ) +} + +struct Rig { + _dir: TempDir, + facade: Arc, +} + +impl Rig { + fn new() -> Self { + let dir = TempDir::new().unwrap(); + let m = Areev::open(dir.path().join("m.db").to_str().unwrap()).unwrap(); + Rig { _dir: dir, facade: Arc::new(AreevFacade::new(m)) } + } + + fn def(&self, t: Tool) -> String { + let t = t.kind(ToolKind::Definition).created_at(500).namespace("ops"); + self.facade.with_store(|m| m.add(&t)).unwrap().to_hex() + } + + fn blob(&self, bytes: &[u8]) -> String { + self.facade.with_store(|m| m.put_blob(bytes)).unwrap() + } + + /// `edges` as (src, dst, cond); `llm` is the plan's `llm` field. + fn plan( + &self, + nodes: &[&str], + edges: &[(&str, &str, Option<&str>)], + bindings: &[(&str, &str)], + llm: Value, + ) -> Hash { + let mut wf = Workflow::new(nodes.iter().map(|s| s.to_string()).collect()); + for (a, b, c) in edges { + wf = match c { + Some(c) => wf.cond_edge(a, b, c), + None => wf.edge(a, b), + }; + } + for (n, h) in bindings { + wf = wf.bind(n, h); + } + let mut wf = wf.created_at(600).namespace("ops"); + if !llm.is_null() { + wf.common.extra_fields.insert("llm".into(), llm); + } + self.facade.with_store(|m| m.add(&wf)).unwrap() + } + + fn runner(&self, exec: Arc, llm: Arc) -> Runner { + Runner { + facade: Arc::clone(&self.facade), + clock: Arc::new(ScriptedClock::new((0..400).map(|i| 1_756_000_000_000 + i * 10).collect())), + executor: exec as Arc, + llm: Some(llm as Arc), + observer: None, + ns: "ops".into(), + principal: "user:runner".into(), + } + } + + fn context(&self, run_id: &str) -> Value { + self.facade + .with_store(|m| areev_run::journal::load(m, "ops", run_id)) + .unwrap() + .checkpoints + .last() + .unwrap() + .scheduler["context"] + .clone() + } + + /// The journaled intent INPUT of every effect of `node`, in key order. + fn intent_inputs(&self, run_id: &str, node: &str) -> Vec<(u32, EffectKind, String, Value)> { + let view = self.facade.with_store(|m| areev_run::journal::load(m, "ops", run_id)).unwrap(); + view.entries + .values() + .filter(|e| e.key.node == node) + .map(|e| { + let g = self.facade.with_store(|m| m.get(&e.intent)).unwrap(); + ( + e.key.effect_seq, + e.key.kind, + g.get_str("tool_name").unwrap_or_default().to_string(), + g.fields.get("input").cloned().unwrap_or(Value::Null), + ) + }) + .collect() + } + + fn manifest_exists(&self, run_id: &str) -> bool { + self.facade.with_store(|m| areev_run::RunManifest::load(m, run_id)).is_ok() + } +} + +fn opts() -> RunOptions { + RunOptions { workers: 2, ..Default::default() } +} + +/// Every run in this file completes; anything else is the failure. +fn finished(s: RunSession) -> RunOutcome { + match s { + RunSession::Finished { outcome, .. } => { + assert_eq!(outcome, RunOutcome::Completed); + outcome + } + other => panic!("expected a finished run, got {other:?}"), + } +} + +/// `lookup → reply (abstract) → gate`, the shape #391 measured: before the +/// block, `reply` was offered `gate` as a tool and saw the whole state. +fn three_step(rig: &Rig, llm: Value) -> Hash { + let lookup = rig.def( + Tool::new("lookup").tool_description("look a record up").input_schema(json!({ + "type": "object", "properties": {"q": {"type": "string"}}, "required": ["q"] + })), + ); + let gate = rig.def(Tool::new("gate").tool_description("final check")); + rig.plan( + &["lookup", "reply", "gate"], + &[("lookup", "reply", None), ("reply", "gate", None)], + &[("lookup", &lookup), ("gate", &gate)], + llm, + ) +} + +// ---- #391 ------------------------------------------------------------------ + +#[test] +fn the_block_is_frozen_sent_journaled_and_replayed_without_the_model() { + let rig = Rig::new(); + let schema = json!({"type": "object", "properties": {"answer": {"type": "string"}}, + "required": ["answer"], "additionalProperties": false}); + let plan = three_step( + &rig, + json!({"reply": { + "system": "You are the accounts-payable assistant.", + "instruction_from": "/prompt", + "state": ["question"], + "tools": ["lookup"], + "response": {"json_schema": schema}, + "temperature": "omit", + "max_tokens": 300, + "parallel_tool_calls": false, + "tool_results_into": "tool_results", + }}), + ); + let exec = Arc::new(TestExec::default()); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "lookup", json!({"q": "4471"}))]), + answer(r#"{"answer": "paid on 2026-10-01"}"#), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + let input = json!({"prompt": "Was invoice 4471 paid?", "question": "4471", "bulk": "x".repeat(400)}); + assert_eq!(finished(runner.start(&plan, "r1", input, &opts()).unwrap()), RunOutcome::Completed); + + // Sent: the node's own system text, no temperature, its ceiling, the + // schema, one call per turn, and ONLY the subset it names. + let sent = llm.sent(); + assert_eq!(sent.len(), 2); + for s in &sent { + assert_eq!(s.system.as_deref(), Some("You are the accounts-payable assistant.")); + assert!(s.omit_temperature, "temperature: omit sends none"); + assert_eq!(s.max_tokens, 300); + assert_eq!(s.parallel_tool_calls, Some(false)); + assert_eq!( + s.response_format, + Some(ResponseFormat::JsonSchema { name: "reply".into(), schema: schema.clone() }) + ); + let names: Vec<&str> = s.tools.iter().map(|(n, _)| n.as_str()).collect(); + assert_eq!(names, vec!["lookup"], "the gate is never offered"); + } + assert_eq!( + sent[0].first_user, + json!({"instruction": "Was invoice 4471 paid?", "state": {"question": "4471"}}), + "the instruction came from the input; the bulky key stayed out of the prompt" + ); + + // The gate below sees what the model read, from the journal. + let gate_input = exec.seen().into_iter().find(|(n, _, _)| n == "gate").unwrap().1; + let log = gate_input["tool_results"].as_array().unwrap(); + assert_eq!(log.len(), 1); + assert_eq!(log[0]["name"], "lookup"); + assert_eq!(log[0]["input"], json!({"q": "4471"})); + assert_eq!(log[0]["result"]["ran"], "lookup"); + assert_eq!(log[0]["is_error"], false); + assert_eq!(log[0]["effect_seq"], 1); + assert_eq!(gate_input["answer"], "paid on 2026-10-01"); + + // Journaled as sent: the mg:llm intent records the system text, the + // instruction and response_format in the wire shape. + let turns: Vec = rig + .intent_inputs("r1", "reply") + .into_iter() + .filter(|(_, k, _, _)| *k == EffectKind::Llm) + .map(|(_, _, name, input)| { + assert_eq!(name, "mg:llm"); + input + }) + .collect(); + assert_eq!(turns[0]["llm"]["system"], "You are the accounts-payable assistant."); + assert_eq!(turns[0]["llm"]["temperature"], "omit"); + assert_eq!(turns[0]["llm"]["response_format"]["type"], "json_schema"); + assert_eq!(turns[0]["llm"]["response_format"]["json_schema"]["schema"], schema); + assert_eq!(turns[0]["messages"][0]["content"]["instruction"], "Was invoice 4471 paid?"); + + // inspect shows the frozen block under the node's pin. + let report = runner.inspect("r1").unwrap(); + let row = report.pinned.iter().find(|r| r["node"] == "reply").unwrap(); + assert_eq!(row["llm"]["tools"], json!(["lookup"])); + assert_eq!(row["llm"]["instruction_from"], "/prompt"); + assert_eq!(row["llm"]["max_tokens"], 300); + assert!(row["llm"].get("narrow").is_none(), "the default rule is stored as nothing"); + + // verify replays byte-identically, never calling the model. + let report = runner.verify("r1").unwrap(); + assert!(report.verified, "{:?}", report.steps); + assert_eq!(llm.sent().len(), 2, "verify never calls the model"); +} + +#[test] +fn a_pinned_tool_the_node_was_not_offered_is_the_unknown_tool_reprompt() { + let rig = Rig::new(); + let plan = three_step(&rig, json!({"reply": {"tools": ["lookup"]}})); + let exec = Arc::new(TestExec::default()); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "gate", json!({}))]), + answer(r#"{"answer": "ok"}"#), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + assert_eq!(finished(runner.start(&plan, "r1", json!({}), &opts()).unwrap()), RunOutcome::Completed); + let sent = llm.sent(); + assert!(sent[1].last_user.contains("Unknown tool"), "{}", sent[1].last_user); + assert!(sent[1].last_user.contains(r#"choose among: ["lookup"]"#), "{}", sent[1].last_user); + assert_eq!( + exec.seen().iter().filter(|(n, _, _)| n == "gate").count(), + 1, + "the gate ran once — as its own step, never as the model's call" + ); + assert!(runner.verify("r1").unwrap().verified); +} + +#[test] +fn a_numeric_temperature_is_sent_and_no_block_sends_what_it_always_did() { + let rig = Rig::new(); + let plan = three_step(&rig, json!({"reply": {"temperature": 0.4}})); + let llm = CapturingLlm::new(vec![answer(r#"{"answer": "ok"}"#)]); + let runner = rig.runner(Arc::new(TestExec::default()), Arc::clone(&llm)); + finished(runner.start(&plan, "r1", json!({}), &opts()).unwrap()); + let s = &llm.sent()[0]; + assert_eq!((s.temperature, s.omit_temperature), (0.4, false)); + assert_eq!(s.max_tokens, 1024, "the run's default ceiling still applies"); + + let plan = three_step(&rig, Value::Null); + let llm = CapturingLlm::new(vec![answer(r#"{"answer": "ok"}"#)]); + let runner = rig.runner(Arc::new(TestExec::default()), Arc::clone(&llm)); + finished(runner.start(&plan, "r2", json!({"q": 1}), &opts()).unwrap()); + let s = &llm.sent()[0]; + assert!(s.system.as_deref().unwrap().starts_with("You are executing one node of a governed workflow.")); + assert_eq!((s.temperature, s.omit_temperature), (0.0, false)); + assert_eq!(s.response_format, None); + assert_eq!(s.parallel_tool_calls, None); + assert_eq!(s.tools.len(), 2, "every pinned host tool, as before"); + assert_eq!(s.first_user["instruction"], "reply"); + assert_eq!(s.first_user["state"]["q"], 1, "the whole state, as before"); + assert_eq!(s.first_user["state"]["ran"], "lookup"); + let turn = &rig.intent_inputs("r2", "reply")[0].3; + assert!(turn.get("llm").is_none(), "no block, nothing new in the journal: {turn}"); + let row = runner.inspect("r2").unwrap().pinned.into_iter().find(|r| r["node"] == "reply").unwrap(); + assert!(row.get("llm").is_none()); +} + +#[test] +fn a_malformed_block_is_refused_at_start_naming_the_node() { + let rig = Rig::new(); + let cases: Vec<(Value, &str, &str)> = vec![ + (json!({"reply": {"tools": ["nope"]}}), "RUN-E004", "'nope'"), + (json!({"lookup": {"system": "x"}}), "RUN-E019", "'lookup'"), + (json!({"ghost": {"system": "x"}}), "RUN-E019", "'ghost'"), + (json!({"reply": {"sytem": "x"}}), "RUN-E019", "\"sytem\""), + (json!({"reply": {"instruction": "a", "instruction_from": "/b"}}), "RUN-E019", "exclusive"), + (json!({"reply": {"instruction_from": "prompt"}}), "RUN-E019", "JSON pointer"), + (json!({"reply": {"state": "some"}}), "RUN-E019", "`state`"), + (json!({"reply": {"response": {"json": false}}}), "RUN-E019", "`response`"), + (json!({"reply": {"response": {"json_schema": {"type": "string"}}}}), "RUN-E019", "json_schema"), + (json!({"reply": {"temperature": "cold"}}), "RUN-E019", "`temperature`"), + (json!({"reply": {"max_tokens": 0}}), "RUN-E019", "`max_tokens`"), + (json!({"reply": {"parallel_tool_calls": "no"}}), "RUN-E019", "parallel_tool_calls"), + (json!({"reply": {"tool_results_into": "$x"}}), "RUN-E019", "tool_results_into"), + (json!({"reply": {"narrow": {"top_k": 0}}}), "RUN-E019", "`narrow`"), + (json!({"reply": {"narrow": "no"}}), "RUN-E019", "`narrow`"), + ]; + for (i, (block, code, needle)) in cases.into_iter().enumerate() { + let plan = three_step(&rig, block.clone()); + let runner = rig.runner(Arc::new(TestExec::default()), CapturingLlm::new(vec![])); + let run_id = format!("bad-{i}"); + let err = runner.start(&plan, &run_id, json!({}), &opts()).unwrap_err(); + assert_eq!(err.code(), code, "{block}: {err}"); + assert!(err.to_string().contains(needle), "{block}: {err}"); + assert!(!rig.manifest_exists(&run_id), "{block}: refused before the run exists"); + } +} + +// ---- #393 ------------------------------------------------------------------ + +#[test] +fn locked_params_win_over_the_model_and_leave_the_offered_schema() { + let rig = Rig::new(); + let post = rig.def( + Tool::new("post") + .tool_description("post a message") + .input_schema(json!({"type": "object", + "properties": {"url": {"type": "string"}, "body": {"type": "string"}}, + "required": ["url", "body"], "additionalProperties": false})) + .strict(true) + .locked_params(json!({"url": "https://a.example/x"})), + ); + // `post` is pinned by a step on a branch that does not fire, so it is + // offered to the agent and never runs as a step of its own. + let done = rig.def(Tool::new("done")); + let plan = rig.plan( + &["agent", "done", "post"], + &[("agent", "done", None), ("agent", "post", Some("never == true"))], + &[("post", &post), ("done", &done)], + Value::Null, + ); + let exec = Arc::new(TestExec::default()); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "post", json!({"url": "https://b.example/y", "body": "hi"}))]), + answer(r#"{"posted": true}"#), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + finished(runner.start(&plan, "r1", json!({}), &opts()).unwrap()); + + let offered = llm.sent()[0].tools.iter().find(|(n, _)| n == "post").unwrap().1.clone().unwrap(); + assert!(offered["properties"].get("url").is_none(), "{offered}"); + assert_eq!(offered["required"], json!(["body"])); + + let (name, input, _) = exec.seen().into_iter().find(|(n, _, _)| n == "post").unwrap(); + assert_eq!(name, "post"); + assert_eq!(input, json!({"url": "https://a.example/x", "body": "hi"}), "the lock wins"); + let tool_intent = rig + .intent_inputs("r1", "agent") + .into_iter() + .find(|(_, k, _, _)| *k == EffectKind::Tool) + .unwrap(); + assert_eq!(tool_intent.3["url"], "https://a.example/x", "the merged input is journaled"); + + let row = runner.inspect("r1").unwrap().pinned.into_iter().find(|r| r["node"] == "post").unwrap(); + assert_eq!(row["locked_params"], json!({"url": "https://a.example/x"})); + assert!(runner.verify("r1").unwrap().verified); +} + +#[test] +fn a_strict_definition_judges_the_merged_input() { + let rig = Rig::new(); + // `body` is required and the model omits it: the merged input still + // lacks it, so the strict check re-prompts — while `url`, required but + // locked, is never reported missing. + let post = rig.def( + Tool::new("post") + .input_schema(json!({"type": "object", + "properties": {"url": {"type": "string"}, "body": {"type": "string"}}, + "required": ["url", "body"]})) + .strict(true) + .locked_params(json!({"url": "https://a.example/x"})), + ); + let done = rig.def(Tool::new("done")); + let plan = rig.plan( + &["agent", "done", "post"], + &[("agent", "done", None), ("agent", "post", Some("never == true"))], + &[("post", &post), ("done", &done)], + Value::Null, + ); + let exec = Arc::new(TestExec::default()); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "post", json!({}))]), + calls(vec![("c2", "post", json!({"body": "hi"}))]), + answer("{\"ok\": true}"), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + finished(runner.start(&plan, "r1", json!({}), &opts()).unwrap()); + let sent = llm.sent(); + assert!(sent[1].last_user.contains("failed validation"), "{}", sent[1].last_user); + assert_eq!( + exec.seen().iter().filter(|(n, _, _)| n == "post").count(), + 1, + "only the corrected call dispatched" + ); + assert!(runner.verify("r1").unwrap().verified); +} + +// ---- #392 ------------------------------------------------------------------ + +/// A plan pinning an `mcp.call`-shaped capability Definition (by a step on a +/// branch that never fires) and an agent whose tools come from the input. +fn mcp_plan(rig: &Rig, llm: Value) -> (Hash, String) { + let uri = rig.blob(b"\0asm-stand-in"); + let mcp = rig.def( + Tool::new("mcp.call") + .tool_description("call a tool on an MCP server") + .input_schema(json!({"type": "object", "properties": { + "url": {"type": "string"}, "tool": {"type": "string"}, + "credential": {"type": "string"}, "arguments": {"type": "object"}}, + "required": ["url", "tool"]})) + .strict(true) + .executor_uri(&uri), + ); + let done = rig.def(Tool::new("done")); + let plan = rig.plan( + &["agent", "done", "mcp.call"], + &[("agent", "done", None), ("agent", "mcp.call", Some("never == true"))], + &[("mcp.call", &mcp), ("done", &done)], + llm, + ); + (plan, mcp) +} + +fn paid_to(definition: &str) -> Value { + json!({ + "name": "paid_to", + "description": "Who an invoice was paid to", + "input_schema": {"type": "object", "properties": {"invoice": {"type": "string"}}, + "required": ["invoice"], "additionalProperties": false}, + "dispatch": {"definition": definition, + "locked": {"url": "https://mcp.example/mcp", "tool": "paid_to", "credential": "gateway"}, + "args_into": "arguments"}, + }) +} + +#[test] +fn dynamic_tools_are_pinned_offered_by_their_own_names_and_dispatched_through_the_definition() { + let rig = Rig::new(); + let (plan, mcp_hash) = mcp_plan(&rig, json!({"agent": {"tools": [], "tools_from": "/tools"}})); + let exec = Arc::new(TestExec { allow_code: true, ..Default::default() }); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "paid_to", json!({"invoice": "4471"}))]), + answer(r#"{"payee": "ACME"}"#), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + let input = json!({"tools": [paid_to("mcp.call")]}); + assert_eq!(finished(runner.start(&plan, "r1", input, &opts()).unwrap()), RunOutcome::Completed); + + // Offered by its own name and schema — and `mcp.call` itself not at all. + let sent = llm.sent(); + assert_eq!(sent[0].tools.len(), 1); + assert_eq!(sent[0].tools[0].0, "paid_to"); + assert_eq!(sent[0].tools[0].1.as_ref().unwrap()["required"], json!(["invoice"])); + + // Executed AS the Definition (its name, its pinned blob), locks set. + let seen = exec.seen(); + assert!(!seen.iter().any(|(n, _, _)| n == "paid_to"), "nothing executes under the dynamic name"); + let (name, input, code) = seen.into_iter().find(|(n, _, _)| n != "done").unwrap(); + assert_eq!(name, "mcp.call", "the host executor and the broker know the Definition's name"); + assert!(code.is_some(), "the pinned blob ran — not --tool-cmd"); + assert_eq!( + input, + json!({"url": "https://mcp.example/mcp", "tool": "paid_to", "credential": "gateway", + "arguments": {"invoice": "4471"}}) + ); + + // Journaled under the dynamic name, with the Definition's address. + let view = rig.facade.with_store(|m| areev_run::journal::load(m, "ops", "r1")).unwrap(); + let entry = view + .entries + .values() + .find(|e| e.key.node == "agent" && e.key.kind == EffectKind::Tool) + .unwrap(); + let g = rig.facade.with_store(|m| m.get(&entry.intent)).unwrap(); + assert_eq!(g.get_str("tool_name"), Some("paid_to")); + assert_eq!(g.get_str("spec_hash"), Some(mcp_hash.as_str())); + let (result, _) = entry.result.as_ref().unwrap(); + let rg = rig.facade.with_store(|m| m.get(result)).unwrap(); + assert_eq!(rg.get_str("tool_name"), Some("paid_to"), "the result re-states the intent's name"); + + // inspect shows the pinned list. + let row = runner.inspect("r1").unwrap().pinned.into_iter().find(|r| r["node"] == "agent").unwrap(); + assert_eq!(row["llm"]["dynamic_tools"][0]["name"], "paid_to"); + assert_eq!(row["llm"]["dynamic_tools"][0]["dispatch"]["definition"], "mcp.call"); + assert_eq!(row["llm"]["dynamic_tools"][0]["dispatch"]["hash"], mcp_hash); + + // verify replays from the journal alone: the model and the tool are + // never called again. + let before = exec.seen().len(); + assert!(runner.verify("r1").unwrap().verified); + assert_eq!(exec.seen().len(), before); + assert_eq!(llm.sent().len(), 2); +} + +#[test] +fn a_dispatch_by_hash_reaches_the_same_definition() { + let rig = Rig::new(); + let (plan, mcp_hash) = mcp_plan(&rig, json!({"agent": {"tools_from": "/tools"}})); + let exec = Arc::new(TestExec { allow_code: true, ..Default::default() }); + let llm = CapturingLlm::new(vec![answer("{}")]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + finished(runner.start(&plan, "r1", json!({"tools": [paid_to(&mcp_hash)]}), &opts()).unwrap()); + let names: Vec = llm.sent()[0].tools.iter().map(|(n, _)| n.clone()).collect(); + assert_eq!(names, vec!["done", "mcp.call", "paid_to"], "no `tools` subset: every pin, then the list"); +} + +#[test] +fn a_bad_tools_list_is_refused_at_start_naming_the_entry() { + let rig = Rig::new(); + let (plan, _) = mcp_plan(&rig, json!({"agent": {"tools_from": "/tools"}})); + let mut dup = paid_to("mcp.call"); + dup["name"] = json!("mcp_call"); + let cases: Vec<(Value, &str, &str)> = vec![ + (json!({"tools": [paid_to("http.call")]}), "RUN-E004", "'http.call'"), + (json!({"tools": [paid_to("mcp.call"), paid_to("mcp.call")]}), "RUN-E019", "duplicate"), + (json!({"tools": [dup]}), "RUN-E019", "collides"), + (json!({}), "RUN-E019", "resolves to nothing"), + (json!({"tools": {"a": 1}}), "RUN-E019", "not a list"), + (json!({"tools": [{"name": "x", "dispatch": {"definition": "mcp.call"}, "inputSchema": {}}]}), + "RUN-E019", "\"inputSchema\""), + (json!({"tools": [{"name": "x y", "dispatch": {"definition": "mcp.call"}}]}), "RUN-E019", "valid tool name"), + (json!({"tools": [{"name": "x", "input_schema": {"type": "array"}, "dispatch": {"definition": "mcp.call"}}]}), + "RUN-E019", "input_schema"), + (json!({"tools": [{"name": "x", "dispatch": {"definition": "mcp.call", "locked": [1]}}]}), + "RUN-E019", "locked"), + ]; + for (i, (input, code, needle)) in cases.into_iter().enumerate() { + let runner = rig.runner(Arc::new(TestExec { allow_code: true, ..Default::default() }), CapturingLlm::new(vec![])); + let run_id = format!("bad-{i}"); + let err = runner.start(&plan, &run_id, input.clone(), &opts()).unwrap_err(); + assert_eq!(err.code(), code, "{input}: {err}"); + assert!(err.to_string().contains(needle), "{input}: {err}"); + assert!(err.to_string().contains("'agent'"), "names the node: {err}"); + assert!(!rig.manifest_exists(&run_id)); + } +} + +#[test] +fn a_dynamic_tools_own_schema_is_enforced_with_a_reprompt() { + let rig = Rig::new(); + let (plan, _) = mcp_plan(&rig, json!({"agent": {"tools_from": "/tools", "tool_results_into": "seen"}})); + let exec = Arc::new(TestExec { allow_code: true, ..Default::default() }); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "paid_to", json!({"invoice": 4471}))]), + calls(vec![("c2", "paid_to", json!({"invoice": "4471"}))]), + answer(r#"{"payee": "ACME"}"#), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + finished(runner.start(&plan, "r1", json!({"tools": [paid_to("mcp.call")]}), &opts()).unwrap()); + assert!(llm.sent()[1].last_user.contains("failed validation")); + assert_eq!(exec.seen().iter().filter(|(n, _, _)| n == "mcp.call").count(), 1); + let ctx = rig.context("r1"); + assert_eq!(ctx["seen"][0]["name"], "paid_to"); + assert_eq!(ctx["seen"][0]["input"]["arguments"], json!({"invoice": "4471"})); + assert_eq!(ctx["seen"][0]["result"]["result"]["payee"], "ACME"); + assert!(runner.verify("r1").unwrap().verified); +} + +#[test] +fn a_resume_uses_the_pinned_block_not_the_live_input() { + // Park on an approval BEFORE the agent runs, then change everything a + // re-resolution would read — supersede the plan with a different block, + // and publish a newer `mcp_call` under the same name — and resume. The + // agent must run the block, the tool list and the Definition frozen at + // start. + let rig = Rig::new(); + let v1_uri = rig.blob(b"\0asm-v1"); + let v1 = Tool::new("mcp_call") + .kind(ToolKind::Definition) + .input_schema(json!({"type": "object", "properties": {"url": {"type": "string"}, + "credential": {"type": "string"}, "tool": {"type": "string"}, "arguments": {"type": "object"}}})) + .executor_uri(&v1_uri) + .created_at(500) + .namespace("ops"); + rig.facade.with_store(|m| m.add(&v1)).unwrap(); + let approve = rig.def(Tool::new("approve").executor_kind(areev_core::types::ExecutorKind::Client)); + let block = |system: &str| json!({"agent": {"system": system, "dispatch": ["mcp_call"], "tools": [], "tools_from": "/tools"}}); + let mut wf = Workflow::new(vec!["approve".into(), "agent".into()]) + .edge("approve", "agent") + .bind("approve", &approve) + .created_at(600) + .namespace("ops"); + wf.common.extra_fields.insert("llm".into(), block("S1")); + let plan = rig.facade.with_store(|m| m.add(&wf)).unwrap(); + + let exec = Arc::new(TestExec { allow_code: true, ..Default::default() }); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "bank_paid_to", json!({"invoice": "4471"}))]), + answer("{}"), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + let RunSession::Parked { envelope, .. } = + runner.start(&plan, "pr1", json!({"tools": [bank_paid_to()]}), &opts()).unwrap() + else { + panic!("parks on the approval") + }; + assert!(llm.sent().is_empty(), "the agent has not run yet"); + + // The live world moves on. + let mut wf2 = Workflow::new(vec!["approve".into(), "agent".into()]) + .edge("approve", "agent") + .bind("approve", &approve) + .created_at(700) + .namespace("ops"); + wf2.common.extra_fields.insert("llm".into(), block("S2")); + rig.facade.with_store(|m| m.supersede(&plan, &mut wf2)).unwrap(); + let v2_uri = rig.blob(b"\0asm-v2"); + let v2 = Tool::new("mcp_call") + .kind(ToolKind::Definition) + .executor_uri(&v2_uri) + .created_at(900) + .namespace("ops"); + rig.facade.with_store(|m| m.add(&v2)).unwrap(); + + let ask = envelope["asks"][0]["tool_call_id"].as_str().unwrap().to_string(); + runner.respond("pr1", &ask, json!({"ok": true}), false, "user:officer").unwrap(); + finished(runner.resume("pr1", &opts()).unwrap()); + + let sent = llm.sent(); + assert_eq!(sent[0].system.as_deref(), Some("S1"), "the frozen block, not the superseding plan's"); + assert_eq!(sent[0].tools[0].0, "bank.paid_to", "the frozen tool list"); + let (_, input, code) = exec.seen().into_iter().find(|(n, _, _)| n == "mcp_call").unwrap(); + assert_eq!(code.as_deref(), Some(v1_uri.as_str()), "the Definition pinned at start, not the newer one"); + assert_eq!(input["arguments"], json!({"invoice": "4471"})); + assert!(runner.verify("pr1").unwrap().verified); +} + +#[test] +fn an_abstract_nodes_call_to_a_code_carrying_definition_runs_its_blob() { + // The lookup a model-issued call is dispatched through used to search by + // NODE, find the abstract node's own pin, and send the call to + // `--tool-cmd`: a capability Definition offered to an agent never ran. + let rig = Rig::new(); + let (plan, _) = mcp_plan(&rig, Value::Null); + let exec = Arc::new(TestExec { allow_code: true, ..Default::default() }); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "mcp_call", json!({"url": "https://mcp.example/mcp", "tool": "t"}))]), + answer("{}"), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + finished(runner.start(&plan, "r1", json!({}), &opts()).unwrap()); + let (_, _, code) = exec.seen().into_iter().find(|(n, _, _)| n == "mcp.call").unwrap(); + assert!(code.is_some(), "ran as its pinned code"); +} + +#[test] +fn every_dispatch_of_a_mixed_run_journals_under_its_kind() { + // Belt and braces for the journal's name rule across all three tool + // shapes in one attempt: a pinned tool, a locked pinned tool and a + // dynamic tool each journal under the name the model called. + let rig = Rig::new(); + let (plan, _) = mcp_plan(&rig, json!({"agent": {"tools_from": "/tools"}})); + let exec = Arc::new(TestExec { allow_code: true, ..Default::default() }); + let llm = CapturingLlm::new(vec![ + calls(vec![ + ("c1", "paid_to", json!({"invoice": "1"})), + ("c2", "mcp_call", json!({"url": "u", "tool": "t"})), + ]), + answer("{}"), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + finished(runner.start(&plan, "r1", json!({"tools": [paid_to("mcp.call")]}), &opts()).unwrap()); + let names: Vec = rig + .intent_inputs("r1", "agent") + .into_iter() + .filter(|(_, k, _, _)| *k == EffectKind::Tool) + .map(|(_, _, n, _)| n) + .collect(); + assert_eq!(names, vec!["paid_to".to_string(), "mcp.call".to_string()]); + assert!(runner.verify("r1").unwrap().verified); +} + +#[test] +fn a_run_using_every_key_at_once_verifies_from_the_journal_alone() { + let rig = Rig::new(); + let uri = rig.blob(b"\0asm-stand-in"); + let mcp = rig.def( + Tool::new("mcp.call") + .input_schema(json!({"type": "object", "properties": { + "url": {"type": "string"}, "tool": {"type": "string"}, "arguments": {"type": "object"}}, + "required": ["url", "tool"]})) + .strict(true) + .executor_uri(&uri) + // The Definition's own lock sits over the entry's. + .locked_params(json!({"credential": "gateway"})), + ); + let post = rig.def( + Tool::new("post") + .input_schema(json!({"type": "object", + "properties": {"url": {"type": "string"}, "body": {"type": "string"}}, + "required": ["url", "body"]})) + .strict(true) + .locked_params(json!({"url": "https://a.example/x"})), + ); + let gate = rig.def(Tool::new("gate")); + let plan = rig.plan( + &["agent", "gate", "mcp.call", "post"], + &[ + ("agent", "gate", None), + ("agent", "mcp.call", Some("never == true")), + ("agent", "post", Some("never == true")), + ], + &[("mcp.call", &mcp), ("post", &post), ("gate", &gate)], + json!({"agent": { + "system": "Persona.", + "instruction_from": "/ask", + "state": ["ask"], + "tools": ["post"], + "tools_from": "/tools", + "response": {"json": true}, + "temperature": "omit", + "max_tokens": 222, + "parallel_tool_calls": true, + "tool_results_into": "read", + }}), + ); + let exec = Arc::new(TestExec { allow_code: true, ..Default::default() }); + let llm = CapturingLlm::new(vec![ + calls(vec![ + ("c1", "paid_to", json!({"invoice": "4471"})), + ("c2", "post", json!({"url": "https://evil.example", "body": "b"})), + ]), + answer("prose first"), + answer(r#"{"payee": "ACME"}"#), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + let input = json!({"ask": "Who was 4471 paid to?", "tools": [{ + "name": "paid_to", + "input_schema": {"type": "object", "properties": {"invoice": {"type": "string"}}, "required": ["invoice"]}, + "dispatch": {"definition": "mcp.call", "locked": {"url": "https://mcp.example/mcp", "tool": "paid_to", + "credential": "the-entry-loses"}, "args_into": "arguments"}, + }]}); + finished(runner.start(&plan, "all-1", input, &opts()).unwrap()); + + let sent = llm.sent(); + assert_eq!(sent.len(), 3, "two turns plus one corrective re-prompt for the prose answer"); + let names: Vec<&str> = sent[0].tools.iter().map(|(n, _)| n.as_str()).collect(); + assert_eq!(names, vec!["post", "paid_to"]); + assert_eq!(sent[0].response_format, Some(ResponseFormat::JsonObject)); + assert_eq!(sent[0].parallel_tool_calls, Some(true)); + assert_eq!(sent[0].max_tokens, 222); + assert!(sent[0].omit_temperature); + let seen = exec.seen(); + let mcp_input = &seen.iter().find(|(n, _, _)| n == "mcp.call").unwrap().1; + assert_eq!(mcp_input["credential"], "gateway", "the Definition's lock wins over the entry's"); + assert_eq!(mcp_input["arguments"], json!({"invoice": "4471"})); + let post_input = &seen.iter().find(|(n, _, _)| n == "post").unwrap().1; + assert_eq!(post_input["url"], "https://a.example/x"); + let gate_input = &seen.iter().find(|(n, _, _)| n == "gate").unwrap().1; + assert_eq!(gate_input["payee"], "ACME"); + let read = gate_input["read"].as_array().unwrap(); + assert_eq!(read.len(), 2); + assert_eq!((read[0]["name"].as_str(), read[1]["name"].as_str()), (Some("paid_to"), Some("post"))); + + let report = runner.verify("all-1").unwrap(); + assert!(report.verified, "{:?}", report.steps); + assert_eq!(llm.sent().len(), 3, "verify never calls the model"); + assert_eq!(exec.seen().len(), seen.len(), "nor any tool"); +} + +#[test] +fn a_parked_run_resumes_under_the_block_it_started_with() { + // An approval first, then the agent: the block the agent runs under after + // a resume is the one frozen at start — whatever input the resume call + // could have carried, it carries none, and the plan is never re-read. + let rig = Rig::new(); + let approve = rig.def(Tool::new("approve").executor_kind(areev_core::types::ExecutorKind::Client)); + let lookup = rig.def(Tool::new("lookup")); + let plan = rig.plan( + &["approve", "agent", "lookup"], + &[("approve", "agent", None), ("agent", "lookup", None)], + &[("approve", &approve), ("lookup", &lookup)], + json!({"agent": {"system": "Frozen persona.", "instruction_from": "/ask", "tools": []}}), + ); + let exec = Arc::new(TestExec::default()); + let llm = CapturingLlm::new(vec![answer(r#"{"done": true}"#)]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + let session = runner.start(&plan, "park-1", json!({"ask": "Reconcile."}), &opts()).unwrap(); + let RunSession::Parked { envelope, .. } = session else { panic!("expected a park") }; + let ask_id = envelope["asks"][0]["tool_call_id"].as_str().unwrap().to_string(); + runner.respond("park-1", &ask_id, json!({"ok": true}), false, "user:officer").unwrap(); + finished(runner.resume("park-1", &opts()).unwrap()); + let sent = llm.sent(); + assert_eq!(sent.len(), 1); + assert_eq!(sent[0].system.as_deref(), Some("Frozen persona.")); + assert_eq!(sent[0].first_user["instruction"], "Reconcile."); + assert!(sent[0].tools.is_empty(), "`tools: []` offers nothing"); + assert!(runner.verify("park-1").unwrap().verified); +} + +// ---- `llm.dispatch`: a Definition no node binds (#392) ---------------------- + +/// The blessed `mcp.call` shape under the name a pack gave it, bound to NO +/// node: only the agent's `dispatch` list pins it. +fn unbound_mcp(rig: &Rig, ns: &str) -> String { + let uri = rig.blob(b"\0asm-mcp"); + let t = Tool::new("mcp_call") + .kind(ToolKind::Definition) + .tool_description("call a tool on an MCP server") + .input_schema(json!({"type": "object", "properties": { + "url": {"type": "string"}, "credential": {"type": "string"}, + "tool": {"type": "string"}, "arguments": {"type": "object"}}, + "required": ["url", "tool", "arguments"], "additionalProperties": false})) + .executor_uri(&uri) + .created_at(500) + .namespace(ns); + rig.facade.with_store(|m| m.add(&t)).unwrap().to_hex() +} + +fn bank_paid_to() -> Value { + json!({ + "name": "bank.paid_to", + "description": "Who an invoice was paid to", + "input_schema": {"type": "object", "properties": {"invoice": {"type": "string"}}, + "required": ["invoice"]}, + "dispatch": {"definition": "mcp_call", + "locked": {"url": "http://127.0.0.1:7798/mcp/orgs/org_x", "credential": "gateway", + "tool": "bank.paid_to"}, + "args_into": "arguments"}, + }) +} + +fn agent_only_plan(rig: &Rig, ns: &str, llm: Value) -> Hash { + let mut wf = Workflow::new(vec!["context".into(), "agent".into()]).edge("context", "agent"); + let ctx = Tool::new("context").kind(ToolKind::Definition).created_at(500).namespace(ns); + let ctx = rig.facade.with_store(|m| m.add(&ctx)).unwrap().to_hex(); + wf = wf.bind("context", &ctx); + let mut wf = wf.created_at(600).namespace(ns); + wf.common.extra_fields.insert("llm".into(), llm); + rig.facade.with_store(|m| m.add(&wf)).unwrap() +} + +#[test] +fn a_dispatch_list_pins_an_unbound_definition_that_is_never_offered_itself() { + let rig = Rig::new(); + let hash = unbound_mcp(&rig, "ops"); + let plan = agent_only_plan( + &rig, + "ops", + json!({"agent": { + "dispatch": ["mcp_call"], + "tools": [], + "tools_from": "/tools", + // A nested pointer: an `op: query` read with FORMAT markdown + // lands `{type: "formatted", text, …}` under its key. + "instruction_from": "/prompt/text", + }}), + ); + let exec = Arc::new(TestExec { allow_code: true, ..Default::default() }); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "bank_paid_to", json!({"invoice": "4471"}))]), + answer(r#"{"payee": "ACME"}"#), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + let input = json!({ + "prompt": {"type": "formatted", "format": "markdown", "text": "## Invoice 4471\nWho was paid?"}, + "tools": [bank_paid_to()], + }); + finished(runner.start(&plan, "d1", input, &opts()).unwrap()); + + let sent = llm.sent(); + assert_eq!(sent[0].first_user["instruction"], "## Invoice 4471\nWho was paid?"); + let names: Vec<&str> = sent[0].tools.iter().map(|(n, _)| n.as_str()).collect(); + assert_eq!(names, vec!["bank.paid_to"], "only the dynamic tool; mcp_call itself is never offered"); + + let (name, input, code) = + exec.seen().into_iter().find(|(n, _, _)| n == "mcp_call").expect("dispatched as mcp_call"); + assert_eq!(name, "mcp_call"); + assert!(code.is_some(), "the pinned blob ran"); + assert_eq!( + input, + json!({"url": "http://127.0.0.1:7798/mcp/orgs/org_x", "credential": "gateway", + "tool": "bank.paid_to", "arguments": {"invoice": "4471"}}), + "{{...locked, args_into: }}" + ); + let journaled = rig + .intent_inputs("d1", "agent") + .into_iter() + .find(|(_, k, _, _)| *k == EffectKind::Tool) + .unwrap(); + assert_eq!(journaled.2, "bank.paid_to"); + assert_eq!(journaled.3, input, "the merged input is what is journaled"); + + let m = rig.facade.with_store(|m| areev_run::RunManifest::load(m, "d1")).unwrap(); + assert_eq!(m.dispatch_pins.len(), 1); + assert_eq!(m.dispatch_pins[0].tool_hash, hash); + assert_eq!(m.dispatch_pins[0].node, "agent"); + assert!(m.dispatch_pins[0].executor_uri.is_some(), "pinned with its code address"); + let report = runner.inspect("d1").unwrap(); + let row = report.pinned.iter().find(|r| r["dispatch_only"] == true).unwrap(); + assert_eq!(row["tool"], "mcp_call"); + assert!(row.get("executor_uri").is_some()); + assert!(runner.verify("d1").unwrap().verified); +} + +#[test] +fn the_merged_input_is_judged_by_the_definitions_schema_strict_or_not() { + // mcp_call is NOT strict, and its schema forbids extra keys: an entry + // that wraps nothing puts the model's `invoice` at the top level, which + // the Definition refuses — so the model gets a correction, and the + // Definition never receives a call it does not take. + let rig = Rig::new(); + unbound_mcp(&rig, "ops"); + let plan = agent_only_plan(&rig, "ops", json!({"agent": {"dispatch": ["mcp_call"], "tools_from": "/tools"}})); + let mut entry = bank_paid_to(); + entry["dispatch"].as_object_mut().unwrap().remove("args_into"); + let exec = Arc::new(TestExec { allow_code: true, ..Default::default() }); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "bank_paid_to", json!({"invoice": "4471"}))]), + answer("{}"), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + finished(runner.start(&plan, "d2", json!({"tools": [entry]}), &opts()).unwrap()); + assert!(llm.sent()[1].last_user.contains("failed validation"), "{}", llm.sent()[1].last_user); + assert!(!exec.seen().iter().any(|(n, _, _)| n == "mcp_call"), "never dispatched"); +} + +#[test] +fn a_dispatch_name_resolves_in_the_plans_namespace_and_a_miss_is_refused() { + let rig = Rig::new(); + // Authored in `lib`, run in `ops`: the plan grain's namespace answers. + unbound_mcp(&rig, "lib"); + let plan = agent_only_plan(&rig, "lib", json!({"agent": {"dispatch": ["mcp_call"], "tools_from": "/tools"}})); + let runner = rig.runner( + Arc::new(TestExec { allow_code: true, ..Default::default() }), + CapturingLlm::new(vec![answer("{}")]), + ); + finished(runner.start(&plan, "d3", json!({"tools": [bank_paid_to()]}), &opts()).unwrap()); + + for (block, input, code, needle) in [ + (json!({"agent": {"dispatch": ["nope"], "tools_from": "/tools"}}), json!({"tools": []}), "RUN-E004", "'nope'"), + (json!({"agent": {"dispatch": "mcp_call"}}), json!({}), "RUN-E019", "`dispatch`"), + // An entry naming a Definition outside the list (and unpinned) is refused. + (json!({"agent": {"tools_from": "/tools"}}), json!({"tools": [bank_paid_to()]}), "RUN-E004", "'mcp_call'"), + ] { + let plan = agent_only_plan(&rig, "lib", block.clone()); + let runner = rig.runner(Arc::new(TestExec { allow_code: true, ..Default::default() }), CapturingLlm::new(vec![])); + let err = runner.start(&plan, "bad", input, &opts()).unwrap_err(); + assert_eq!(err.code(), code, "{block}: {err}"); + assert!(err.to_string().contains(needle), "{block}: {err}"); + assert!(!rig.manifest_exists("bad")); + } +} + +#[test] +fn an_unpinned_dispatch_blob_is_refused_before_the_run_exists() { + let rig = Rig::new(); + unbound_mcp(&rig, "ops"); + let plan = agent_only_plan(&rig, "ops", json!({"agent": {"dispatch": ["mcp_call"], "tools_from": "/tools"}})); + let runner = rig.runner(Arc::new(TestExec::default()), CapturingLlm::new(vec![])); + let err = runner.start(&plan, "d4", json!({"tools": [bank_paid_to()]}), &opts()).unwrap_err(); + assert_eq!(err.code(), "RUN-E018", "{err}"); + assert!(!rig.manifest_exists("d4")); +} + +#[test] +fn a_dynamic_schema_carrying_annotation_keywords_is_pinned_and_dispatched() { + // #397: what an MCP server's tools/list publishes, annotations and all. + let rig = Rig::new(); + unbound_mcp(&rig, "ops"); + let plan = agent_only_plan(&rig, "ops", json!({"agent": {"dispatch": ["mcp_call"], "tools": [], "tools_from": "/tools"}})); + let mut entry = bank_paid_to(); + entry["input_schema"] = json!({ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "urn:bank:paid_to", "$comment": "generated", + "title": "paid_to", "description": "arguments", + "type": "object", + "properties": {"invoice": {"type": "string", "title": "Invoice", "description": "number", + "examples": ["4471"], "default": "0", "deprecated": false, + "readOnly": false, "writeOnly": false}}, + "required": ["invoice"], + }); + let exec = Arc::new(TestExec { allow_code: true, ..Default::default() }); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "bank_paid_to", json!({"invoice": "4471"}))]), + answer("{}"), + ]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + finished(runner.start(&plan, "a1", json!({"tools": [entry]}), &opts()).unwrap()); + let offered = llm.sent()[0].tools[0].1.clone().unwrap(); + for k in ["$schema", "$id", "$comment"] { + assert!(offered.get(k).is_none(), "{k} is ignored, not offered: {offered}"); + } + assert!(offered["properties"]["invoice"].get("readOnly").is_none()); + assert_eq!(offered["properties"]["invoice"]["title"], "Invoice", "portable annotations stay"); + let (_, input, _) = exec.seen().into_iter().find(|(n, _, _)| n == "mcp_call").unwrap(); + assert_eq!(input["arguments"], json!({"invoice": "4471"})); + assert!(runner.verify("a1").unwrap().verified); +} + +// ---- review fixes ------------------------------------------------------------ + +#[test] +fn a_flat_entry_with_a_required_locked_key_is_offered_and_dispatched_without_it() { + // Locks without `args_into`: the entry requires `url`, the host locks it. + let rig = Rig::new(); + unbound_mcp(&rig, "ops"); + let http = rig.def(Tool::new("http_get").input_schema(json!({"type": "object", + "properties": {"url": {"type": "string"}, "q": {"type": "string"}}, "required": ["url"]}))); + let plan = agent_only_plan(&rig, "ops", json!({"agent": {"dispatch": [http.clone()], "tools": [], "tools_from": "/tools"}})); + let entry = json!({"name": "search", "input_schema": {"type": "object", + "properties": {"url": {"type": "string"}, "q": {"type": "string"}}, "required": ["url", "q"]}, + "dispatch": {"definition": "http_get", "locked": {"url": "https://a.example/s"}}}); + let exec = Arc::new(TestExec::default()); + let llm = CapturingLlm::new(vec![calls(vec![("c1", "search", json!({"q": "x"}))]), answer("{}")]); + let runner = rig.runner(Arc::clone(&exec), Arc::clone(&llm)); + finished(runner.start(&plan, "f1", json!({"tools": [entry]}), &opts()).unwrap()); + let offered = llm.sent()[0].tools[0].1.clone().unwrap(); + assert_eq!(offered["required"], json!(["q"]), "{offered}"); + assert!(offered["properties"].get("url").is_none()); + assert_eq!(llm.sent().len(), 2, "no re-prompt: the call the model was shown how to make is valid"); + let (_, input, _) = exec.seen().into_iter().find(|(n, _, _)| n == "http_get").unwrap(); + assert_eq!(input, json!({"q": "x", "url": "https://a.example/s"})); +} + +#[test] +fn a_lock_on_the_args_into_key_or_a_dotted_lock_is_refused_at_start() { + let rig = Rig::new(); + unbound_mcp(&rig, "ops"); + let plan = agent_only_plan(&rig, "ops", json!({"agent": {"dispatch": ["mcp_call"], "tools_from": "/tools"}})); + let mut on_args = bank_paid_to(); + on_args["dispatch"]["locked"]["arguments"] = json!({}); + let mut dotted = bank_paid_to(); + dotted["dispatch"]["locked"]["arguments.invoice"] = json!("1"); + for (entry, needle) in [(on_args, "`args_into` key"), (dotted, "dotted")] { + let runner = rig.runner(Arc::new(TestExec { allow_code: true, ..Default::default() }), CapturingLlm::new(vec![])); + let err = runner.start(&plan, "bad", json!({"tools": [entry]}), &opts()).unwrap_err(); + assert_eq!(err.code(), "RUN-E019", "{err}"); + assert!(err.to_string().contains(needle) && err.to_string().contains("bank.paid_to"), "{err}"); + } +} + +#[test] +fn a_response_schema_strict_mode_would_refuse_is_refused_at_start() { + let rig = Rig::new(); + for (schema, needle) in [ + (json!({"type": "object", "properties": {"a": {"type": "string"}}, "required": ["a"]}), "additionalProperties"), + (json!({"type": "object", "properties": {"a": {"type": "string"}, "b": {"type": "string"}}, + "required": ["a"], "additionalProperties": false}), "'b'"), + (json!({"type": "object", "properties": {"n": {"type": "object", "properties": {"x": {"type": "string"}}, + "required": ["x"]}}, "required": ["n"], "additionalProperties": false}), "/properties/n"), + ] { + let plan = three_step(&rig, json!({"reply": {"response": {"json_schema": schema}}})); + let runner = rig.runner(Arc::new(TestExec::default()), CapturingLlm::new(vec![])); + let err = runner.start(&plan, "bad", json!({}), &opts()).unwrap_err(); + assert_eq!(err.code(), "RUN-E019", "{err}"); + assert!(err.to_string().contains("strict mode") && err.to_string().contains(needle), "{err}"); + } +} + +#[test] +fn a_dispatch_pin_may_not_be_another_version_of_a_bound_definition() { + let rig = Rig::new(); + let v1 = rig.def(Tool::new("lookup").input_schema(json!({"type": "object", "properties": {"q": {"type": "string"}}}))); + let v2 = { + let t = Tool::new("lookup") + .kind(ToolKind::Definition) + .input_schema(json!({"type": "object", "properties": {"z": {"type": "string"}}})) + .created_at(900) + .namespace("ops"); + rig.facade.with_store(|m| m.add(&t)).unwrap().to_hex() + }; + let gate = rig.def(Tool::new("gate")); + let plan = rig.plan( + &["lookup", "agent", "gate"], + &[("lookup", "agent", None), ("agent", "gate", None)], + &[("lookup", &v1), ("gate", &gate)], + json!({"agent": {"dispatch": [v2.clone()]}}), + ); + let runner = rig.runner(Arc::new(TestExec::default()), CapturingLlm::new(vec![])); + let err = runner.start(&plan, "bad", json!({}), &opts()).unwrap_err(); + assert_eq!(err.code(), "RUN-E019", "{err}"); + assert!(err.to_string().contains(&v2) && err.to_string().contains("'lookup'"), "{err}"); +} + +#[test] +fn locked_params_on_a_client_tool_are_ignored_with_a_warning_not_refused() { + // 1.12 accepted this shape; a minor release must keep running it. + let rig = Rig::new(); + let approve = rig.def( + Tool::new("approve") + .executor_kind(areev_core::types::ExecutorKind::Client) + .locked_params(json!({"level": "senior"})), + ); + let plan = rig.plan(&["approve"], &[], &[("approve", &approve)], Value::Null); + let runner = rig.runner(Arc::new(TestExec::default()), CapturingLlm::new(vec![])); + let session = runner.start(&plan, "w1", json!({}), &opts()).unwrap(); + assert!(matches!(session, RunSession::Parked { .. })); + let report = runner.inspect("w1").unwrap(); + assert_eq!(report.warnings.len(), 1, "{:?}", report.warnings); + assert!(report.warnings[0].starts_with("RUN-W001") && report.warnings[0].contains("approve")); + let m = rig.facade.with_store(|m| areev_run::RunManifest::load(m, "w1")).unwrap(); + assert!(m.pinned[0].locked_params.is_none()); +} + +#[test] +fn shadow_rehearses_with_the_locks_the_incumbent_froze_not_the_definitions() { + // A run recorded WITHOUT locks (any 1.12 run): the strict Definition + // requires `url`, so the model's first call {body} was re-prompted and + // its second, with a url, dispatched. Rehearsing it with the Definition's + // locks would accept the first call and diverge on an unchanged plan. + let rig = Rig::new(); + let approve = rig.def(Tool::new("approve").executor_kind(areev_core::types::ExecutorKind::Client)); + let post = rig.def( + Tool::new("post") + .input_schema(json!({"type": "object", + "properties": {"url": {"type": "string"}, "body": {"type": "string"}}, + "required": ["url", "body"]})) + .strict(true) + .locked_params(json!({"url": "https://a.example/x"})), + ); + let done = rig.def(Tool::new("done")); + let plan = rig.plan( + &["approve", "agent", "done", "post"], + &[("approve", "agent", None), ("agent", "done", None), ("agent", "post", Some("never == true"))], + &[("approve", &approve), ("post", &post), ("done", &done)], + Value::Null, + ); + let llm = CapturingLlm::new(vec![ + calls(vec![("c1", "post", json!({"body": "hi"}))]), + calls(vec![("c2", "post", json!({"url": "https://b.example", "body": "hi"}))]), + answer("{}"), + ]); + let runner = rig.runner(Arc::new(TestExec::default()), Arc::clone(&llm)); + let RunSession::Parked { envelope, .. } = runner.start(&plan, "old", json!({}), &opts()).unwrap() else { + panic!("parks on the approval") + }; + // Make the frozen manifest the 1.12 shape: no locks. + let mut m = rig.facade.with_store(|m| areev_run::RunManifest::load(m, "old")).unwrap(); + for p in &mut m.pinned { + p.locked_params = None; + } + rig.facade.with_store(|mm| m.persist_in_namespace(mm, "ops")).unwrap(); + let ask = envelope["asks"][0]["tool_call_id"].as_str().unwrap().to_string(); + runner.respond("old", &ask, json!({"ok": true}), false, "user:officer").unwrap(); + finished(runner.resume("old", &opts()).unwrap()); + assert_eq!(llm.sent().len(), 3, "recorded without locks: one re-prompt"); + + let report = runner.shadow_plan(&["old".to_string()], &areev_run::PlanCandidate::Hash(plan)).unwrap(); + let run = &report.runs[0]; + assert_eq!(run.verdict, "same", "{run:?}"); + assert!(run.out_of_support.is_empty(), "{run:?}"); + assert!(run.identity.as_ref().is_some_and(|i| i.consistent), "{run:?}"); +} + +#[test] +fn the_result_records_the_temperature_that_went_out_not_the_one_asked_for() { + // A transport whose profile pins 0.7, under a node asking 0.2. + struct Pinned(Arc); + impl ToolCallLlm for Pinned { + fn model(&self) -> &str { + "m" + } + fn effective_temperature(&self, _requested: f64) -> Option { + Some(0.7) + } + fn request_profile_digest(&self) -> Option { + Some("pinned".into()) + } + fn call(&self, req: &ToolCallRequest<'_>) -> Result { + self.0.call(req) + } + } + let rig = Rig::new(); + let plan = three_step(&rig, json!({"reply": {"temperature": 0.2}})); + let inner = CapturingLlm::new(vec![answer("{}")]); + let runner = Runner { llm: Some(Arc::new(Pinned(inner)) as Arc), ..rig.runner(Arc::new(TestExec::default()), CapturingLlm::new(vec![])) }; + finished(runner.start(&plan, "t1", json!({}), &opts()).unwrap()); + let view = rig.facade.with_store(|m| areev_run::journal::load(m, "ops", "t1")).unwrap(); + let e = view.entries.values().find(|e| e.key.node == "reply").unwrap(); + let areev_run_core::EffectOutcome::Completed { result, .. } = &e.result.as_ref().unwrap().1 else { panic!() }; + assert_eq!(result["temperature_sent"], json!(0.7)); + + // No block and no profile: nothing new on the result. + let plan = three_step(&rig, Value::Null); + let llm = CapturingLlm::new(vec![answer("{}")]); + finished(rig.runner(Arc::new(TestExec::default()), llm).start(&plan, "t2", json!({}), &opts()).unwrap()); + let view = rig.facade.with_store(|m| areev_run::journal::load(m, "ops", "t2")).unwrap(); + let e = view.entries.values().find(|e| e.key.node == "reply").unwrap(); + let areev_run_core::EffectOutcome::Completed { result, .. } = &e.result.as_ref().unwrap().1 else { panic!() }; + assert!(result.get("temperature_sent").is_none(), "{result}"); +} + +#[test] +fn two_dispatch_pins_of_one_name_and_different_versions_are_refused() { + let rig = Rig::new(); + let mk = |at: i64, prop: &str| { + let t = Tool::new("mcp_call") + .kind(ToolKind::Definition) + .input_schema(json!({"type": "object", "properties": {prop: {"type": "string"}}})) + .created_at(at) + .namespace("ops"); + rig.facade.with_store(|m| m.add(&t)).unwrap().to_hex() + }; + let (a, b) = (mk(500, "x"), mk(510, "y")); + for block in [ + json!({"agent": {"dispatch": [a.clone(), b.clone()]}}), + json!({"agent": {"dispatch": [a.clone()]}, "agent2": {"dispatch": [b.clone()]}}), + ] { + let ctx = rig.def(Tool::new("context")); + let mut wf = Workflow::new(vec!["context".into(), "agent".into(), "agent2".into()]) + .edge("context", "agent") + .edge("context", "agent2") + .bind("context", &ctx) + .created_at(600) + .namespace("ops"); + wf.common.extra_fields.insert("llm".into(), block.clone()); + let plan = rig.facade.with_store(|m| m.add(&wf)).unwrap(); + let runner = rig.runner(Arc::new(TestExec::default()), CapturingLlm::new(vec![])); + let err = runner.start(&plan, "bad", json!({}), &opts()).unwrap_err(); + assert_eq!(err.code(), "RUN-E019", "{block}: {err}"); + assert!(err.to_string().contains(&a) && err.to_string().contains(&b), "{err}"); + } +} diff --git a/crates/areev-run/tests/memory_read_tests.rs b/crates/areev-run/tests/memory_read_tests.rs index 109d9fd4..d40bb7bd 100644 --- a/crates/areev-run/tests/memory_read_tests.rs +++ b/crates/areev-run/tests/memory_read_tests.rs @@ -568,6 +568,7 @@ fn the_tool_pool_never_answers_a_memory_read() { idempotency_key: "k".into(), llm: None, code: None, + exec_name: None, }); let done = pool.done_rx.recv().unwrap(); let EffectOutcome::Failed { cause, detail, .. } = done.outcome else { diff --git a/crates/areev-server/Cargo.toml b/crates/areev-server/Cargo.toml index e5fd8c10..0422dd46 100644 --- a/crates/areev-server/Cargo.toml +++ b/crates/areev-server/Cargo.toml @@ -13,14 +13,14 @@ keywords = ["memory", "server", "sync", "ai-agents", "http"] categories = ["web-programming::http-server"] [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } -areev-store = { path = "../areev-store", version = "1.12.0" } -areev-cal = { path = "../areev-cal", version = "1.12.0" } -areev-loop = { path = "../areev-loop", version = "1.12.0" } -areev-run = { path = "../areev-run", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } +areev-store = { path = "../areev-store", version = "1.13.0" } +areev-cal = { path = "../areev-cal", version = "1.13.0" } +areev-loop = { path = "../areev-loop", version = "1.13.0" } +areev-run = { path = "../areev-run", version = "1.13.0" } rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"], optional = true } rustls-pki-types = { version = "1.15", features = ["std"], optional = true } -areev-loop-adapter = { path = "../areev-loop-adapter", version = "1.12.0" } +areev-loop-adapter = { path = "../areev-loop-adapter", version = "1.13.0" } serde_json = "1" # Native OIDC (`oidc` feature) — the SECOND recorded dependency-policy # exception, after rustls. Signature verification against an issuer-published diff --git a/crates/areev-server/src/console.html b/crates/areev-server/src/console.html index d9c6268b..c49f8440 100644 --- a/crates/areev-server/src/console.html +++ b/crates/areev-server/src/console.html @@ -4163,7 +4163,8 @@ one through the console would silently unbound its loop. A plan that declares memory `reads` (#255) opens view-only for the same reason: CAL has no syntax for them, and a resave would turn each read step into an - abstract LLM step. */ + abstract LLM step. A per-node `llm` block (#391) likewise: a resave would + drop it, and with it the node's prompt, tool subset and schema. */ function workflowGrains() { return wfRows().filter(g => g.type === 'workflow' && isCurrent(g)); } function toolDefs() { return wfRows().filter(g => g.type === 'tool' && (g.fields || {}).kind === 'definition' && isCurrent(g)); } function wfLabel(g) { @@ -4178,6 +4179,7 @@ if (edges.some(e => e.max_cycles != null)) return false; if (f.retries && Object.keys(f.retries).length) return false; if (f.reads && Object.keys(f.reads).length) return false; + if (f.llm && Object.keys(f.llm).length) return false; return true; } /* `reads` wins over `bindings`: the runtime refuses a node that declares both, diff --git a/crates/areev-store/Cargo.toml b/crates/areev-store/Cargo.toml index 088c3740..c621e8bd 100644 --- a/crates/areev-store/Cargo.toml +++ b/crates/areev-store/Cargo.toml @@ -38,7 +38,7 @@ postgres-tls = [ ] [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } # Pinned exactly: encryption-at-rest rides turso's experimental AES-256-GCM, # so the audited version is fixed until a deliberate, tested bump. turso = "=0.7.2" diff --git a/crates/areev-trigger/Cargo.toml b/crates/areev-trigger/Cargo.toml index f5b4457e..f3f7f95f 100644 --- a/crates/areev-trigger/Cargo.toml +++ b/crates/areev-trigger/Cargo.toml @@ -13,10 +13,10 @@ keywords = ["ai-agents", "memory", "scheduling", "triggers", "oms"] categories = ["database"] [dependencies] -areev-core = { path = "../areev-core", version = "1.12.0" } -areev-store = { path = "../areev-store", version = "1.12.0" } -areev-cal = { path = "../areev-cal", version = "1.12.0" } -areev-run = { path = "../areev-run", version = "1.12.0" } +areev-core = { path = "../areev-core", version = "1.13.0" } +areev-store = { path = "../areev-store", version = "1.13.0" } +areev-cal = { path = "../areev-cal", version = "1.13.0" } +areev-run = { path = "../areev-run", version = "1.13.0" } serde = { version = "1", features = ["derive"] } serde_json = "1" sha2 = "0.11" @@ -47,4 +47,4 @@ tz = ["dep:chrono-tz"] [dev-dependencies] tempfile = "3" -areev-run-core = { path = "../areev-run-core", version = "1.12.0" } +areev-run-core = { path = "../areev-run-core", version = "1.13.0" } diff --git a/docs/assets/repo-stats-dark.svg b/docs/assets/repo-stats-dark.svg index c2cf61b0..fb7a9c1f 100644 --- a/docs/assets/repo-stats-dark.svg +++ b/docs/assets/repo-stats-dark.svg @@ -1,16 +1,16 @@ - + Areev — repository quality -v1.12.2 · 17 crates · 356 Rust files +v1.13.0 · 17 crates · 358 Rust files -116,840 +118,059 source code -74,143 +75,723 test code -3,366 +3,409 tests 83.6% @@ -19,9 +19,9 @@ 128 error codes - -tests 38.8% -source 61.2% -44,096 integration · 30,047 unit test lines · 66 reference docs · 17 crates + +tests 39.1% +source 60.9% +45,464 integration · 30,259 unit test lines · 66 reference docs · 17 crates Lines exclude blanks and comments; test code counted per block, not per file. Coverage scores source lines only, floored per crate. diff --git a/docs/assets/repo-stats-light.svg b/docs/assets/repo-stats-light.svg index bc93c4a8..c6794868 100644 --- a/docs/assets/repo-stats-light.svg +++ b/docs/assets/repo-stats-light.svg @@ -1,16 +1,16 @@ - + Areev — repository quality -v1.12.2 · 17 crates · 356 Rust files +v1.13.0 · 17 crates · 358 Rust files -116,840 +118,059 source code -74,143 +75,723 test code -3,366 +3,409 tests 83.6% @@ -19,9 +19,9 @@ 128 error codes - -tests 38.8% -source 61.2% -44,096 integration · 30,047 unit test lines · 66 reference docs · 17 crates + +tests 39.1% +source 60.9% +45,464 integration · 30,259 unit test lines · 66 reference docs · 17 crates Lines exclude blanks and comments; test code counted per block, not per file. Coverage scores source lines only, floored per crate. diff --git a/docs/blessed-tools.md b/docs/blessed-tools.md index ec67db0f..7fa94253 100644 --- a/docs/blessed-tools.md +++ b/docs/blessed-tools.md @@ -145,6 +145,39 @@ them. endpoint receiving `text/plain` answers 415 — and a declaration must name every header it sets. Not supported in v1: the SSE/streamable transport. +**Behind `tools_from` (#392).** Offered directly to a model, `mcp.call` is one +opaque tool. The model picks the server URL and the MCP tool name, and the +host cannot pin which server or which tool a call reaches. To offer an MCP +server's tools one by one, under their own names and schemas, pin the +Definition with the abstract node's `llm.dispatch` and list the tools in the +run input. Each entry dispatches through `mcp.call` with the server, the MCP +tool and the credential locked, and the model's arguments wrapped under +`arguments`: + +```json +"llm": {"reply": {"dispatch": ["mcp.call"], "tools": [], "tools_from": "/tools"}} +``` + +```json +{"tools": [{"name": "bank.paid_to", "description": "Who an invoice was paid to", + "input_schema": {"type": "object", "properties": {"invoice": {"type": "string"}}, + "required": ["invoice"]}, + "dispatch": {"definition": "mcp.call", + "locked": {"url": "https://mcp.example.com/mcp", "tool": "paid_to", + "credential": "mcp"}, + "args_into": "arguments"}}]} +``` + +The model sees `bank_paid_to` and never `mcp.call`. A call is validated +against the entry's schema. The input `{url, tool, credential, arguments}` is +then built and validated against `mcp.call`'s own `input_schema`, and that +input is what this blob receives. It runs as `mcp.call`, under the same +`--allow-executor` pin and the same broker grants (`--tool-egress +'mcp.call:mcp@mcp.example.com:POST'`), and it is journaled under +`bank.paid_to`. Without `args_into`, the model's arguments would land at the +top level, beside `url`, and the blob would send `arguments: {}`. See +[`docs/run.md`](run.md#tools-declared-in-the-run-input-tools_from). + ### `a2a.call` The same envelope with `message/send` and the A2A `params` shape: diff --git a/docs/repo-stats.html b/docs/repo-stats.html index afb696fb..e6fe537c 100644 --- a/docs/repo-stats.html +++ b/docs/repo-stats.html @@ -57,15 +57,15 @@

Areev — repository statistics

-

v1.12.2 · generated from the tree by +

v1.13.0 · generated from the tree by scripts/repo_stats.py

-
74,143
lines of test code
-
116,840
lines of source code
-
3,366
test functions
+
75,723
lines of test code
+
118,059
lines of source code
+
3,409
test functions
83.6%
line coverage (source only)
-
0.63:1
test to source
+
0.64:1
test to source
17
workspace crates
128
stable error codes
@@ -77,20 +77,20 @@

By crate

CoverageFloor areev-cal5026,76821,5191,10673.0%70% + areev-run3410,47012,54134888.9%79% areev-store4513,4827,47537285.6%83% - areev-run329,85911,54132588.9%79% - areev-cli259,1179,22325576.7%70% + areev-cli259,1509,24225676.7%70% areev-loop3815,6962,18227194.7%90% areev-core529,8405,03133785.2%80% areev-bench237,1212,869115not scored— + areev-run-core83,4132,2287092.4%89% + areev-llm113,1432,44611581.3%73% areev-trigger132,1402,72014189.8%89% - areev-llm112,9422,23410781.3%73% - areev-run-core83,0711,8795992.4%89% areev-context92,1352,99511383.6%79% areev-conformance293,1781,5683597.6%95% areev-server32,3101,7268382.3%75% - areev-js23,53500not scored— - areev-py23,01400not scored— + areev-js23,55000not scored— + areev-py23,03100not scored— areev-loop-adapter91,2111,1814784.6%78% areev-mcp11,3640076.6%71% @@ -126,14 +126,14 @@

Composition

- - - - - + + + + + - - + +
Rust files356
Physical lines (all Rust)252,894
Source code (no blanks or comments)116,840
Unit test code (#[cfg(test)] blocks)30,047
Integration test code (tests/, benches/)44,096
Rust files358
Physical lines (all Rust)256,359
Source code (no blanks or comments)118,059
Unit test code (#[cfg(test)] blocks)30,259
Integration test code (tests/, benches/)45,464
Example code (examples/)669
Test functions3,366
Reference docs66 files, 36,847 lines
Test functions3,409
Reference docs66 files, 37,168 lines
Stable error codes128
diff --git a/docs/repo-stats.json b/docs/repo-stats.json index f6c66949..0080126f 100644 --- a/docs/repo-stats.json +++ b/docs/repo-stats.json @@ -1,17 +1,17 @@ { - "version": "1.12.2", + "version": "1.13.0", "crates": 17, "rust": { - "files": 356, - "physical_lines": 252894, - "source_code": 116840, - "test_code": 74143, - "unit_test_code": 30047, - "integration_test_code": 44096, + "files": 358, + "physical_lines": 256359, + "source_code": 118059, + "test_code": 75723, + "unit_test_code": 30259, + "integration_test_code": 45464, "example_code": 669, - "test_functions": 3366, - "test_ratio": 0.63, - "test_share": 38.8 + "test_functions": 3409, + "test_ratio": 0.64, + "test_share": 39.1 }, "per_crate": [ { @@ -22,6 +22,14 @@ "test_code": 21519, "tests": 1106 }, + { + "name": "areev-run", + "files": 34, + "physical": 30448, + "source_code": 10470, + "test_code": 12541, + "tests": 348 + }, { "name": "areev-store", "files": 45, @@ -30,21 +38,13 @@ "test_code": 7475, "tests": 372 }, - { - "name": "areev-run", - "files": 32, - "physical": 28525, - "source_code": 9859, - "test_code": 11541, - "tests": 325 - }, { "name": "areev-cli", "files": 25, - "physical": 23189, - "source_code": 9117, - "test_code": 9223, - "tests": 255 + "physical": 23249, + "source_code": 9150, + "test_code": 9242, + "tests": 256 }, { "name": "areev-loop", @@ -71,28 +71,28 @@ "tests": 115 }, { - "name": "areev-trigger", - "files": 13, - "physical": 6979, - "source_code": 2140, - "test_code": 2720, - "tests": 141 + "name": "areev-run-core", + "files": 8, + "physical": 7622, + "source_code": 3413, + "test_code": 2228, + "tests": 70 }, { "name": "areev-llm", "files": 11, - "physical": 6741, - "source_code": 2942, - "test_code": 2234, - "tests": 107 + "physical": 7271, + "source_code": 3143, + "test_code": 2446, + "tests": 115 }, { - "name": "areev-run-core", - "files": 8, - "physical": 6734, - "source_code": 3071, - "test_code": 1879, - "tests": 59 + "name": "areev-trigger", + "files": 13, + "physical": 6979, + "source_code": 2140, + "test_code": 2720, + "tests": 141 }, { "name": "areev-context", @@ -121,16 +121,16 @@ { "name": "areev-js", "files": 2, - "physical": 4833, - "source_code": 3535, + "physical": 4865, + "source_code": 3550, "test_code": 0, "tests": 0 }, { "name": "areev-py", "files": 2, - "physical": 4289, - "source_code": 3014, + "physical": 4321, + "source_code": 3031, "test_code": 0, "tests": 0 }, @@ -153,7 +153,7 @@ ], "docs": { "files": 66, - "lines": 36847 + "lines": 37168 }, "error_codes": 128, "bindings": { diff --git a/docs/repo-stats.md b/docs/repo-stats.md index 853dc1e6..60b7d50e 100644 --- a/docs/repo-stats.md +++ b/docs/repo-stats.md @@ -2,36 +2,36 @@ -v1.12.2 · 17 crates · 356 Rust files +v1.13.0 · 17 crates · 358 Rust files | | | |---|---| -| Source code | **116,840** lines | -| Test code | **74,143** lines (38.8% of all code) | -| Test functions | **3,366** | +| Source code | **118,059** lines | +| Test code | **75,723** lines (39.1% of all code) | +| Test functions | **3,409** | | Line coverage | **83.6%** of 76,211 instrumented source lines | | Stable error codes | **128** | -| Reference docs | 66 files, 36,847 lines | +| Reference docs | 66 files, 37,168 lines | ## By crate | Crate | Files | Source | Test | Tests | |---|---:|---:|---:|---:| | `areev-cal` | 50 | 26,768 | 21,519 | 1,106 | +| `areev-run` | 34 | 10,470 | 12,541 | 348 | | `areev-store` | 45 | 13,482 | 7,475 | 372 | -| `areev-run` | 32 | 9,859 | 11,541 | 325 | -| `areev-cli` | 25 | 9,117 | 9,223 | 255 | +| `areev-cli` | 25 | 9,150 | 9,242 | 256 | | `areev-loop` | 38 | 15,696 | 2,182 | 271 | | `areev-core` | 52 | 9,840 | 5,031 | 337 | | `areev-bench` | 23 | 7,121 | 2,869 | 115 | +| `areev-run-core` | 8 | 3,413 | 2,228 | 70 | +| `areev-llm` | 11 | 3,143 | 2,446 | 115 | | `areev-trigger` | 13 | 2,140 | 2,720 | 141 | -| `areev-llm` | 11 | 2,942 | 2,234 | 107 | -| `areev-run-core` | 8 | 3,071 | 1,879 | 59 | | `areev-context` | 9 | 2,135 | 2,995 | 113 | | `areev-conformance` | 29 | 3,178 | 1,568 | 35 | | `areev-server` | 3 | 2,310 | 1,726 | 83 | -| `areev-js` | 2 | 3,535 | 0 | 0 | -| `areev-py` | 2 | 3,014 | 0 | 0 | +| `areev-js` | 2 | 3,550 | 0 | 0 | +| `areev-py` | 2 | 3,031 | 0 | 0 | | `areev-loop-adapter` | 9 | 1,211 | 1,181 | 47 | | `areev-mcp` | 1 | 1,364 | 0 | 0 | @@ -39,12 +39,12 @@ v1.12.2 · 17 crates · 356 Rust files | | | |---|---:| -| Physical lines (all Rust) | 252,894 | -| Source code (no blanks or comments) | 116,840 | -| Unit test code (`#[cfg(test)]` blocks) | 30,047 | -| Integration test code (`tests/`, `benches/`) | 44,096 | +| Physical lines (all Rust) | 256,359 | +| Source code (no blanks or comments) | 118,059 | +| Unit test code (`#[cfg(test)]` blocks) | 30,259 | +| Integration test code (`tests/`, `benches/`) | 45,464 | | Example code (`examples/`) | 669 | -| Test functions | 3,366 | +| Test functions | 3,409 | ## Coverage diff --git a/docs/run.md b/docs/run.md index c6d5addb..77727dc8 100644 --- a/docs/run.md +++ b/docs/run.md @@ -127,6 +127,13 @@ What each piece means: (`entity_at`, `related`, `recall`, or a pinned saved `query`). The runtime answers that node itself, from the file it already holds; no tool is involved. See [Reading the run's own memory](#reading-the-runs-own-memory-reads). +- **`llm`**: node → the configuration of an **abstract** node's model loop + (#391). It sets the system text, the instruction (literal or from the + input), which state keys the model sees, which pinned tools it is offered, + tools declared in the run input (`tools_from`, #392), structured output, + temperature, the output ceiling, and whether the node's tool results reach + the nodes below. It is frozen at start like `reads`. See + [Configuring an LLM node](#configuring-an-llm-node-llm). Plan validation runs at load: unreachable nodes (`RUN-E003`), unresolvable bindings (`RUN-E004`), malformed conditions (`RUN-E005`), unbounded cycles @@ -1606,6 +1613,254 @@ areev run start --workflow --run-id r1 --input '{}' \ missing key fails without leaving behind a run that could never advance. The backend is host config and is deliberately not journaled with the run, which is why `run_resume` takes it too rather than recovering it from the manifest. +- **Locked parameters** (#393). A Definition's `locked_params` are keys the + model must not choose. When an abstract node's model calls the tool, the + runtime merges them over the model's arguments at the top level, and the lock + wins. The intent journals the merged input and the executor receives it. The + transcript keeps what the model wrote. The locked keys are removed from the + `input_schema` the model is offered, from `properties` and from `required`. + A `strict` Definition validates the merged input, so a required key that is + locked is never reported missing. The locks are frozen into the run manifest + with the rest of the pin (`run inspect` shows them as `locked_params`). A run + that started before its Definition locked anything never starts locking + mid-run, and `shadow` rehearses a run with the locks that run froze, not + the Definition's. + **Where locks apply:** only in run dispatch (`areev-run`), when an + abstract node's model calls a tool. `areev-loop`, `areev-server` (the + console) and `areev-mcp` do not dispatch model-issued tool calls, so they do + not apply locks. Neither does a bound host node, whose input is the run's + state and not a model's choice. A client or decision Definition that + declares `locked_params` is not refused: no model calls it, so the locks are + ignored, and the start records `RUN-W001` in the manifest's `warnings` + (shown by `run inspect`). +- **From Python and Node**, `run_start`/`runStart` and `run_resume`/`runResume` + also take `llm_profile`/`llmProfile` as their **last** argument (#394). It is + a JSON string that shapes every request the transport sends: + `{"temperature": "omit" | , "token_field": "max_tokens" | + "max_completion_tokens", "extra_body": {…}}`, with every key optional. It is + the bindings' form of the CLI's `--llm-no-temperature`, `--llm-token-field` + and `--llm-extra-body` (`$AREEV_RUN_LLM_NO_TEMPERATURE`, + `$AREEV_RUN_LLM_TOKEN_FIELD`, `$AREEV_RUN_LLM_EXTRA_BODY`). `triggerRun`/ + `triggerDeliver` and `trigger_run`/`trigger_deliver` take it last too, and + `areev eval --model` reads the CLI flags. When a binding is given no + `llm_profile`, it reads those three variables, as the CLI does. + `false`, `0`, `off` and `no` all mean off, and a bare + `--llm-no-temperature` takes no value. An unknown key, + a profile with no `model`, and a profile on the Ollama transport (whose body + takes none of it) are refused before a run exists. The run pins the + profile's digest (#287), so a resume passes the same profile or is refused + with `RUN-E025`. For one node, the plan's `llm` block below does the same job + and needs no host configuration. + +### Configuring an LLM node (`llm`) + +By default an abstract node gets a fixed system prompt, the node name as its +instruction and the whole merged state. It is offered every pinned host tool, +asks for no structured output and runs at temperature 0. A plan's `llm` field +changes any of that per node (#391). It is a sibling of `reads`, `reducers` and +`retries`. The runtime validates it at `run start` and freezes it into the +abstract node's pin in the run manifest. A resume, a fork of the same plan and +`verify` all run the frozen block, never the plan as it stands later. + +```json +{ + "nodes": ["context", "reply", "gate"], + "edges": [{"src": "context", "dst": "reply"}, {"src": "reply", "dst": "gate"}], + "bindings": {"gate": ""}, + "reads": {"context": {"op": "query", "name": "account_brief", + "params_from": {"account": "/account"}, "into": "prompt"}}, + "llm": { + "reply": { + "system": "You are the accounts-payable assistant for this organisation.", + "instruction_from": "/prompt/text", + "state": ["question", "account"], + "tools": ["records_query"], + "dispatch": ["mcp_call"], + "tools_from": "/tools", + "response": {"json_schema": {"type": "object", + "properties": {"answer": {"type": "string"}, "sources": {"type": "array"}}, + "required": ["answer", "sources"], "additionalProperties": false}}, + "temperature": "omit", + "max_tokens": 2048, + "parallel_tool_calls": false, + "tool_results_into": "tool_results", + "narrow": {"top_k": 12} + } + } +} +``` + +| Key | Value | Effect | +|---|---|---| +| `system` | string | Replaces the built-in system prompt for this node's turns, its fold summaries included. | +| `instruction` | string | The `instruction` of `messages[0]`. Default: the node name. | +| `instruction_from` | JSON pointer | Takes the instruction from the node's input. The pointer may be nested: an `op: query` read with `FORMAT markdown` lands `{"type": "formatted", "text": …}`, so `/prompt/text` reaches the text. If the pointer resolves to nothing, or to something that is not a string, the node fails `schema_validation_failed` before any model call. It is not retried, because the same state fails the same way. Exclusive with `instruction`. | +| `state` | `"all"` \| `"none"` \| `[keys]` | Which merged-state keys `messages[0].state` carries. Default `"all"`. A steering `$inbox` (`run input`) is always carried, because an operator's message must reach the model. Under `"all"`, the value `tools_from` points at is left out, along with the locks inside it: it is configuration, not state. | +| `tools` | `[names]` | The subset of the plan's pinned host Definitions offered, in manifest order. Default: all. Accepts the canonical or the provider-safe spelling (`a.b` or `a_b`). A name the plan does not pin is refused at start with `RUN-E004`, naming the node and the name. A pinned tool the node was not offered is unknown to the model: calling it gets the unknown-tool re-prompt. `[]` offers nothing. | +| `dispatch` | `[name \| hash]` | Definitions this node's dynamic tools dispatch through that no node binds (see `tools_from`). | +| `tools_from` | JSON pointer | Tools declared in the run input (see below). | +| `response` | `{"json_schema": {…}}` \| `{"json": true}` | Structured output, described below. | +| `temperature` | number \| `"omit"` | Replaces the fixed 0. `"omit"` sends no temperature key at all, which the GPT-5 family needs: it answers HTTP 400 to any value. | +| `max_tokens` | integer ≥ 1 | This node's output ceiling per call. Replaces `llm_max_tokens` and lands in the profile's token field (`max_tokens` or `max_completion_tokens`). It is also the turn's per-dispatch reservation against the token budget. | +| `parallel_tool_calls` | boolean | Sent as `parallel_tool_calls` on OpenAI-compatible endpoints. Anthropic receives the inverse, `tool_choice.disable_parallel_tool_use`. Ollama ignores it. When absent, nothing is sent and the provider's default applies. | +| `narrow` | `true` \| `false` \| `{"top_k": n}` | How a calibrated decision backend narrows this node's offer (see [Narrowing the tool offer](#narrowing-the-tool-offer)). `false` keeps every tool, `{"top_k": n}` keeps the top `n` (1–254) instead of 8, and `true` (the default) is the run-wide rule. | +| `tool_results_into` | state key | Adds this key to the node's result when the node completes. It holds every tool call of the attempt, in `effect_seq` order: `[{name, tool_call_id, model_call_id, input, result, is_error, effect_seq}]`. `tool_call_id` is the journal key's digest (the grain's `tool_call_id`), `input` is the dispatched input with locks merged, and `result` is the whole journaled result, never the transcript's bounded excerpt. A gate bound below the node therefore sees what the model read. The list is built from the journaled outcomes, so `verify` rebuilds it byte for byte. If the model's answer has a key with the same name, the runtime's value wins. | + +**`system` and the model boundary.** Under an `egress` anonymization policy, +the node's `system` text is pseudonymized as its messages are. So are a +dynamic tool's description and the prose of its schema (`description`, +`title`, `default`, `examples`). Structural schema strings and the response +schema go out as declared, because arguments and answers are checked against +them as declared. + +**What is journaled.** The first message is `messages[0]` of every turn's +intent. The request fields the block sets go into each `mg:llm` intent's +`input.llm`, as sent: `{system, temperature, max_tokens, parallel_tool_calls, +response_format}`. `response_format` is always in the OpenAI-compatible wire +shape (`{"type": "json_schema", "json_schema": {"name": "", "schema": …, +"strict": true}}` or `{"type": "json_object"}`). The driver builds each request +from that journaled input alone. The plan's `temperature` is a request, and +a transport's request profile may pin or drop it. So whenever a turn carries +a block or the transport a non-default profile, the result grain records +`temperature_sent`, the value that actually went out (`null` for none). +`gen_ai.request.temperature` reports the same value. A node with no block, or +with an empty one, on a transport with no profile, journals exactly what it +did before 1.13. + +**Structured output.** On OpenAI-compatible endpoints `response` is sent as +`response_format`. The wire `name` is the node id with characters outside +`[a-zA-Z0-9_-]` replaced by `_`. Because OpenAI's strict mode applies, the +schema must satisfy it: every object sets `additionalProperties: false` and +lists every property in `required`. A schema that does not is refused at +start with `RUN-E019`, naming the first violation by JSON pointer. +`{"json": true}` is sent as `json_object`, which OpenAI refuses unless a +message says "JSON". When the node's own `system` text and its instruction +both lack the word, the runtime appends "Answer with a JSON object." to the +system text, and the journaled `llm.system` includes that sentence. Ollama receives the schema as `format`. +Anthropic has no such field, so the schema is appended to the system text. On +every provider the runtime then checks the final answer itself: it must be a +JSON object and, with `json_schema`, conform to it. An answer wrapped in a +Markdown code fence (```` ```json … ``` ````, as Anthropic models often send) +is unwrapped first, and it merges unwrapped. The schema must be in the +subset a Tool's `input_schema` is held to (an object root with no +`oneOf`/`$ref`), or the plan is refused at start. A violation gets **one** +corrective re-prompt. The model's own answer stays in the transcript so it can +see what it said. A second violation fails the node with +`schema_validation_failed`. A conforming answer merges into state as any JSON +object answer does. + +Like `reads`, an `llm` block is authored through the generic JSON `add`, +because CAL `ADD workflow` has no syntax for it. The console opens a plan that +has one view-only, since a resave would drop the block. + +**Refused at start**, before the run exists, naming the node: an unknown key, +a block on a node that is not abstract or not in the plan, both `instruction` +and `instruction_from`, a pointer that does not start with `/`, a malformed +`state`, `response`, `temperature`, `max_tokens`, `parallel_tool_calls`, +`tool_results_into` or `narrow` (`RUN-E019`), and a `tools` or `dispatch` name the run +cannot resolve (`RUN-E004`). + +#### Tools declared in the run input (`tools_from`) + +The tools a node may call are normally fixed when the plan is authored. A host +whose agent must call tools that differ per tenant and per day, such as the +`tools/list` of an MCP server the tenant connected, puts them in the run input +instead (#392). `tools_from` is a JSON pointer into the **run input** that +names a list of entries: + +```json +{"tools": [{ + "name": "bank.paid_to", + "description": "Who an invoice was paid to", + "input_schema": {"type": "object", "properties": {"invoice": {"type": "string"}}, + "required": ["invoice"]}, + "dispatch": { + "definition": "mcp_call", + "locked": {"url": "http://127.0.0.1:7798/mcp/orgs/org_x", "credential": "gateway", + "tool": "bank.paid_to"}, + "args_into": "arguments" + } +}]} +``` + +- At `run start` the list is resolved and **pinned into the manifest** with + the node's block: names, descriptions, schemas, the dispatch Definition's + name and content address, its `input_schema`, the locks and `args_into`. A + resume and `verify` offer exactly that set, whatever input a later call + carries. `run inspect` shows it under the node's `llm.dynamic_tools`. +- The node is offered each entry **by its own name and schema**, after its + `tools` subset. Dots become underscores on the wire, as for any Definition, + and a call to either spelling resolves. +- **`dispatch.definition`** names the Definition the call reaches, by + `tool_name` or content address. It must be in the node's `dispatch` list or + be a host tool the plan pins. A Definition that is neither is refused at + start (`RUN-E004`, naming the entry). A dynamic tool may only reach code the + plan's reviewer saw. +- **`llm..dispatch: ["mcp_call"]`** pins a Definition that no node binds, + such as the blessed `mcp.call` under whatever name the pack gave it. Each + entry, a name or a 64-hex content address, resolves at start exactly as an + unbound node's same-named Definition does: in the run's namespace, then in + the plan grain's namespace. A miss is `RUN-E004` naming the entry. The + Definition is pinned like a bound one, with its code address, runtime, + capabilities and locks, and the start refuses an unpinned code address + (`RUN-E018`) as it does for a node. It is **never offered to the model by + itself**. Only the dynamic tools that dispatch through it are offered. + `run inspect` lists it among the pins with `"dispatch_only": true`. It may + not be another version of a Definition a node binds under the same + `tool_name` (`RUN-E019`, naming both hashes). +- **Without `args_into`, the entry's schema is closed.** The model's + arguments then land at the top level of the Definition's input. A key the + entry does not name is refused, so a model cannot set a Definition + parameter the host never offered (such as a `method` on an http-shaped + Definition that locks only the URL). A key the host **locked** is refused + too, rather than silently overridden. **With `args_into`**, the arguments + belong to the remote tool, and the entry's own schema decides: it is closed + only by `additionalProperties: false`. An MCP tool published as + `{"type": "object"}` takes any key. Every refusal is an argument violation, + and the model is re-prompted. Locks set **top-level** keys of the Definition's input only: + a dotted lock key, or a lock key equal to `args_into` (which would replace + the model's arguments), is refused at start with `RUN-E019`, naming the + entry. That applies to the entry's `locked` and to the Definition's own + `locked_params`. +- **A call** is validated against the entry's `input_schema`, always. The + model's arguments are then placed under `args_into` when the entry names + one (`mcp.call` takes the MCP tool's arguments one level down, under + `arguments`), or at the top level when it does not. Then `locked` is merged + over them, then the Definition's own `locked_params` over that, and the lock + wins each time. Without `args_into`, the model's arguments are checked + against the schema it was shown, with locked keys stripped from + `properties` and `required`, so a required key the host locks is never + demanded of it. The merged input `{...locked, [args_into]: }` is + validated against the dispatch Definition's `input_schema`, whether or not + the Definition is `strict`. A violation at either step is a corrective + re-prompt, as for any tool. The merged input is what the intent journals and + what the Definition receives. +- **The call executes as the Definition**: the host executor, the sandbox and + the broker's `--tool-egress` grants all see the Definition's name, and its + pinned blob runs. The grain is journaled under the dynamic tool's name, with + the Definition's address as `spec_hash`. `verify` replays it from the + journal alone. +- **Refused at start**, naming the entry (`RUN-E019`): a pointer that resolves + to nothing or to a non-list, an entry that is not an object or carries an + unknown key, an invalid name, a duplicate name (including two names that + differ only by `.`/`_`), a name that collides with a tool the plan pins, an + `input_schema` outside the tool-schema subset, and a `locked` that is not an + object. +- **Annotation keywords are accepted and ignored** (#397), at every level of + an entry's `input_schema`, of `response.json_schema`, and of a Definition's + `input_schema`/`output_schema` when it is written. The keywords are + `$schema`, `$id`, `$comment`, `deprecated`, `readOnly`, `writeOnly` and + vendor `x-*` keys. They are removed before the subset check, so the pinned, + offered and validating schema is the clean form. `title`, `description`, + `examples` and `default` are annotations too: they are already in the + subset and are kept. The structural rule is unchanged: `$ref`, `oneOf`, + `anyOf`, `allOf` and `patternProperties` are still refused. + +A model-issued call from an abstract node reaches the Definition it was +offered as **by its frozen content address**. Before 1.13 the driver looked a +call's code up by node. It found the abstract node's own pin, which names no +code, so a call to a code-carrying Definition, a capability tool among them, +fell through to `--tool-cmd` instead of running its pinned blob. ### What bounds the transcript (and what does not) @@ -1939,6 +2194,15 @@ shown is unknown to it: calling one is the unknown-tool re-prompt. An uncalibrated answer, a failure or a table that does not cover every tool leaves the full offer. +A node's `llm` block can change this per node (#396). `"narrow": false` keeps +that node's whole offer, and nothing is asked. `"narrow": {"top_k": n}` (1–254) +narrows to the top `n` instead of 8, and is asked only when more than `n` +tools are offered. The p ≥ 0.05 rule still applies. `"narrow": true`, or no +key at all, is the rule above. The setting is frozen in the manifest with the +rest of the block. A dynamic tool (`tools_from`) is described in the question +by the first line of its own `description`, the same way a pinned +Definition's `tool_description` is. + The narrowed set rides every turn's journaled input as `offer` — `{tools, seq, provider, model, calibrated, latency_ms}`. The driver offers exactly those Definitions, keyed off the journal the way the summarizer's