Repository navigation
Release 0.74.0 — the instruments stop flattering the product - #20
Merged
Merged
Conversation
J.5.15 drew the path an answer took and said, in its own changelog, what it
could not do: a walk's hops arriving one at a time, live, needs the host to
push. J.10.12 is that push, and it is smaller than the estimate because the
estimate had the wrong cause.
A walk holds its reader lane for its whole duration (J.10.11 says so), and
the assumption was that emitting a hop mid-call therefore needed J.10.11's
treatment first — the model lifted off the lane. It does not. A lane is a
thread in an executor and the event loop is reachable from any thread, so a
hop can be handed across without the walk changing where it runs at all. The
lane question is real, it is about throughput, and conflating it with this
one would have bought a large refactor to ship a small feature.
What the section adds is one route and one optional field. A caller may put
an opaque `run` on a POST to `answer`; a caller that does not is byte-for-
byte where it was, response included. `GET .../answer/{run}/events` carries
the same call's progress: `retrieval` when the sweep's bundle exists, `hop`
as each of a walk's steps completes, `done` at the close.
The rule that makes it safe is the one that also makes it simple, and it is
deliberately NOT J.16's. A webhook leaves the Station's authority behind —
whoever holds the URL reads it — so J.16 rations its payload down to
identity. This stream is pulled, by the same principal, under the same
credential and the same scope, for a call that principal is already making.
So the rule is not "carry less than the response"; it is carry nothing the
completed response would not have carried to this same principal. An event
is a PREFIX of the answer, never a second disclosure surface, and F.138
checks that by comparison rather than by a list of allowed fields.
Three properties keep a spectator from costing the answer anything: emission
never blocks, nothing outlives its call, and the stream is never the answer.
Rule 6 gained a bounded claim grace after the first end-to-end run failed on
it: a watcher must be free to open the channel BEFORE firing the call, since
awaiting the POST means awaiting the whole answer, so an unclaimed run is a
race rather than an absence. Bounded is the requirement — a channel closes,
it never hangs.
Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
A served forest of about twelve hundred nodes was asked what it was about.
In walk mode it read the readme and stopped; in sweep mode it generalised
five ranked excerpts into a claim about the whole forest. Neither answer is
a hallucination and neither is a defect in the loop — both are this
specification working exactly as written, which is why the repair is four
spec sentences and not a patch.
The walk's tool menu was closed ("and nothing else") around every primitive
except the one built for that question class: `coverage` is on it now. It
widens nothing — metadata only, scope-filtered like every other read — and a
tool the model is never offered is a tool the whitelist did not admit.
The sweep's prompt had no denominator. `searched` rides the empty path only,
which is C.1.1's rule and stays, so five items of twelve hundred arrive in
the exact shape twelve hundred of twelve hundred would, and generalising
them is prompt-COMPLIANT rather than prompt-defying. The prompt must state
that the material is a ranked top-k of a larger corpus — J.10.8's "the cap
is said whatever chose it", applied to the sample size.
The walk's entry is a synthetic `locate` of the question verbatim, and the
prompt never said so, so re-authoring retrieval — translating a Portuguese
question into an English corpus's vocabulary, reaching for the rarer term —
read as repeating a move already made rather than as a first one. The
machinery was always there; the sentence was not. Wording stays
implementation freedom; the omissions do not.
`answer` gains `terms` (sweep only; `terms` with `hops` is E_SCHEMA, because
a walk authors its own retrieval and a parameter silently dropped is a lie
about what ran). It rides C.6c's sniff leg on harvest's own validation and
enters the J.10.7 key as the effective terms — which makes that key's
"whether the caller supplied them or the sweep derived them" true for the
first time since v0.33. No planning turn is added: C.6c stays zero-LLM,
J.10.11's phases stay ordered, and a call without `terms` keys identically.
The J.5.15 panel drew what did not happen, twice. On a walk it fired a
`harvest` the walk never runs and painted it as the answer's retrieval, so
the wrong picture arrived first and the live hops displaced it — which reads
as the model ignoring what it was shown. The preview is mode-aware now, the
walk's panel starts empty (an empty stage is a fact) and fills from J.10.12
`hop` events; for those to light anything the hop record gains `ids`, since
a count lights no node. The background is dots-only: it had been painting
the whole J.11 edge set including the `confidence<1` class Explore hides by
default, at twice structural opacity.
The derived-term stopword set also gains the PT/ES demonstratives. `esta`
was absent while `this` and `that` were present, so a Portuguese question
sniffed as a substring hit inside `restart`.
Acceptance F.139-F.142. F.142 is a static source check: the canvas is
unverifiable (F.137's boundary), so dots-only, colour and legend are
normative text with no test, and this commit says so rather than implying
coverage it does not have.
Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
… as the corpus The implementation of J.10.12 and of v0.67's four repairs. They arrive as one commit because they are one state: v0.67 amends the very panel v0.66 feeds, and no point ever existed where the first stood alone and passed. **The progress channel (J.10.12).** `runs.py` is the host-memory board, keyed by (principal, forest, run) — the key IS the authorization, since a record exists only because that principal's own `answer` on that forest claimed it. `publish` is called from a forest lane and the consumer lives on the event loop, which is exactly the case an asyncio primitive cannot serve, so the lock is a plain `threading.Lock` and the wake is `call_soon_threadsafe`. It never awaits and never raises: a spectator does not get a vote on whether the hunt continues, and a buffer that fills drops and COUNTS rather than applying back-pressure. `forage` gains `on_hop`, called with the record it just appended — the same object the response will carry as `hops[n]`, because an event is a prefix of the answer and a second rendering of it would be a different claim. The sweep publishes its bundle where the bundle is computed, which is the same object the response carries as `harvest`. F.138 compares the two rather than asserting each, which is the only way that rule is checkable. `run` is popped in dispatch beside `hybrid`, for the same reason: it is the host's field and not the primitive's, so what `validate_args` checks is what the composite reads. The MCP surface gains nothing. **The client reads the channel with `fetch`, not `EventSource`,** for one reason that decides it: an EventSource cannot carry a header, and the alternative is the key in the query string — a credential in a URL is a credential in every log and every referrer. So the SSE frames are parsed in `api.js`, which is a dozen lines. The J.5.15 pre-flight retrieval survives as a fallback, held back behind a grace window rather than raced: on a Station that serves the channel the console now runs no second retrieval at all, and on one that does not the map still draws. **v0.67's half:** `coverage` on the walk's whitelist and in its stated menu; the sweep's prompt stating that its material is a ranked top-k; the walk's prompt stating that its entry was a synthetic `locate` of the question verbatim and routing corpus-scope questions to structure; `answer` accepting `terms` and forwarding them to the sniff leg; `ids` on the hop record so a live event can light a node; the mode-aware panel with zoom, pan and a dots-only background; and the PT/ES demonstratives in the derived-term stopword set. `trailmap.js` holds what the panel decides — which node belongs to which stage, which segments the trail draws — separated from `trail.jsx` for the reason `skill.js` is separated from `Skills.jsx`: a criterion inside a React component is a criterion no machine can ask about. `check-trail.mjs` puts F.137's and F.139-F.141's questions to it and runs from pytest. Verified end to end against a real provider, not only the suite: a three-hop walk published its hops at second 1, 3 and 4 of a five-second call, and the events matched the response's own `hops` field for field. Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
An operator asked a hosted question with hybrid entry on and read, on the very panel J.10.4 exists for: forest 8437 ms, of which `locate` was 8391 — beside sniffs and picks in fractions of a millisecond. The panel's own premise is that "the forest is slow" is almost always the wrong suspicion, and here the panel itself was raising it: the forest's whole work in that trace was some forty milliseconds. The other 8.3 seconds were the K.2 query embed — one HTTP round trip to the embedding provider, run inside `locate`'s span because K.2 puts it there, and timed as `locate` because Part D times a primitive from outside, as it should. Nothing sums the answer model into the retrieval figure — J.10.4 has reported those apart since it existed. But the other model on the read path, the embedder, had no clock of its own, so its round trip is billed to whichever primitive it ran inside: `locate` when the entry embeds the query, a walk's first hop when the goal embeds lazily. The figure indicts the wrong half, and the K.6 memo makes it unreproducible: the second ask of the same question embeds from `_derived`, so the number vanishes on exactly the retry meant to confirm it. The repair is a named share, never a second stopwatch. The Part D event gains `embed_ms` — memo hits included, absent when no embed ran, every other event byte-identical. `elapsed_ms` stays the whole wall span: it is true, and a total that quietly excluded a slow provider would be the flattering version of the same lie. J.10.4 forwards the field on the step that paid it and sums it once as `trace.embed_ms`, present only when nonzero; `retrieval_ms`, `total_ms` and J.10.6's header are unchanged to the byte. The console rule: every step and the forest figure are shown net of their share, and the summed share is listed once at the tail beside the `model` step, in the model's own tone — provider spend sits with provider spend, and every primitive row is the engine's own smallest true number. Acceptance: F.143–F.144; the panel's rendering is normative text with no test, on F.142's stated boundary. Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
… true number The implementation of spec v0.68. The engine collects the K.2/K.6 embed's cost where it runs: `Vine.embed_query` accumulates into `_embed_ms_pending` — memo hit and round trip alike, because a hit's near-zero is the memo working and only the named share can say so — and `_traced` resets and reads it around every call (save/restore, so a traced call inside another keeps its own share) before recording it as `embed_ms` on the Part D event. A call that ran no embed emits the v0.67 event to the byte, and the lazily embedded goal lands the field on the hop that paid it, never on the `locate` that deferred it. The host forwards the field per step and sums `trace.embed_ms` only when nonzero. `retrieval_ms`, `total_ms` and Server-Timing are untouched: the host serves the whole span plus the named share, never a subtracted figure — the netting is the console's. The console nets it. `TraceSteps` prints every primitive row at the engine's own work (its ms less its share) and lists the sum once, as a synthetic `embed` row at the tail beside `model`, in the model's tone — provider spend beside provider spend. Built in the shared component, so Ask and the Playground agree. The Ask cards read forest (net) / embedding / model, and the locale files name the middle card in all three languages. F.143 (tests/test_embed_clock.py, on test_gauntlet's deterministic embedder — a SlowEmbedder sleep makes the round trip distinguishable from the memo beside it) and F.144 (tests/test_station_explain.py, an answer through a real embedder binding and canopy; the step-shape guard admits `embed_ms` beside the old five and nothing else). Suite 1384 green; Studio builds. Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
Spec v0.69, v0.70 and v0.71, landing together because they share one file: `vine.py` carries all three and splitting it by hunk would produce commits that never ran as written. v0.69 — a path is not a syscall. `Forest.path_for` mapped an id to a file and decided containment in the same breath, with `Path.resolve()`: a realpath walk, one syscall per component, called once per node in scope by the body scan. 72 ms of a ~230 ms cold `sniff` over 1,877 nodes, against 2.6 ms for the same containment decided on the string. The split is by PROVENANCE and never by cost: an id arriving from outside the engine — the wire, `ScopedVine`, host-supplied paths, and every write, whose id came from a caller — resolves symlinks; an id read from this forest's own catalog takes `path_for(..., trusted=True)`, where normpath still collapses `..` so traversal is still refused and only symlink-following is given up. Keyword-only and never dispatched. The rule is only as good as its boundary list because the failure is silent — nothing raises, nothing logs, an id just passes — so `test_containment.py` is a surfaces x escapes matrix and a surface absent from it is not covered. `_fold_table` also gains a lock (N readers of the J.6.2 pool could each build their own copy) and is built in `warm()`, never at import. v0.70 — the question is not the query. `locate` handed the whole sentence to FTS5, so every article and preposition was a search term with a vote, while `harvest` has derived terms since C.6c and the entry search it calls did not. Measured on a 60-question set with headroom: recall@1 0.711 -> 0.778, MRR 0.734 -> 0.791, five questions up and none down, with p50 latency 1.9 -> 0.7 ms as a side effect. The load-bearing half is the fallback: `api`, `sql`, `ui` and a sentence made of grammar all derive to nothing, and passing that through would answer nothing at all to a caller who typed one word — C.1.1's failure, manufactured. An empty derivation searches the raw query and those calls stay byte-identical. Stated cost, not left to be rediscovered: "erro sql no worker" searches `erro`+`worker` and drops the discriminating term; lowering the floor to 3 changed nothing on the labelled set, so it was left alone rather than tuned against no evidence. v0.71 — the scan is the other half of the bill. v0.68 named the provider round trip and closed the case on half of it. The same query asked twice separates them: cold, `locate` raw 262 ms with embed 166; warm, the K.6 memo answers in 0.13 ms and 74 ms is still charged to `locate`. It is not the network. `CanopyIndex.search` over 1,877 vectors of dim 1024 measures 68 ms — 36 us per vector — against 0.35 ms for the catalog refill and 0.01 for the fusion. v0.42's "the vector scan was never the problem (0.044 ms per dim-1024 dot product)" was a true claim about ONE comparison, and a hybrid entry makes one per node: it survives its own arithmetic only while the corpus is small. `dense_ms` now rides the Part D event beside `embed_ms`, saved and restored across nested traced calls. They stay apart because they are repaired apart — whoever is dominated by `embed_ms` buys a closer embedder, whoever is dominated by `dense_ms` needs an index instead of a scan. Order note, recorded rather than hidden: v0.71's measurement had to come before its contract, so the code preceded that spec cut. The rule is still spec-before-code. Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
The three labelled sets this project carried scored recall@1 = 1.000 (fixture, 10 questions), 1.000 (wide-forest, 24) and 0.778 (bench-forest, 18). Two at the ceiling and one that resolves a single question, so no change to the entry ranker could be approved or refused. That is not a methodology footnote: it is why the grammar-filtering change measured 3 up and 3 down before this set existed, and 5 up and 0 down after. `questions-locate-v1` is 75 questions over a 1,877-node Portuguese corpus, five classes of fifteen, baseline recall@1 0.711 on unmodified code: code_lookup the query IS an identifier (DT-038, INC-004) inflection the question uses a form the document does not index_vs_content the target sits under an `_index` that matches as well grammar written as a person speaks; the worst class at 0.533 silence expected_nodes: [], the forest genuinely lacks it Two rules it was written under, because a set that breaks them is worse than none. `expected_nodes` never came from running a search — documents were chosen by structural position, read, and the question written from what the document says; a set labelled from search output scores 1.000 by construction, which is the likeliest explanation for the two saturated sets above. And no question was tuned after seeing its score. `bench_locate.py` reads the class tag, prints per-class recall, scores the silence class apart (a silence and a hit are not the same event and averaging them hides both), and skips with exit 0 when the forest is absent — the corpus is not in this repository and CI must not break on that. What the set found on its first run, neither of which is fixed here: `locate` returned results for all 15 questions the forest cannot answer, and `strength` is normalised against the best hit in its own result set, so the top scores 1.000 for every question — the absolute BM25 that does separate them (silence median -11.5 against -24..-39) is discarded before the caller sees it. That is C.1.1 from the other side and wants its own round. And what it refused: the prefix leg, drafted before it existed. Measured per class the leg leaves every content class unchanged at every threshold and moves only the one it was meant to protect, 0.067 -> 0.000. It fires 0/15 on the inflection class it was built for, 13/15 on code lookups where the exact query already scored 1.000, and 5/15 on silences. Narrowed to fire only on an empty result it fires once in 75, and that once turns "aquisicao de licencas do AutoCAD para engenharia" — correctly nothing — into three results, one of them `022-auto-refresh-token`, because AutoCAD truncated to `auto*` matches an auth token refresher. The rescue and the damage are the same operation, and no floor separates them. Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
Both found by pressing them, which is the whole point: neither had ever
been exercised end to end.
`POST /v1/admin/recurate` failed on every call from the console with
`AttributeError: 'str' object has no attribute 'get'`. The cause is in the
client: `request()` already stringifies the body, and `recurate` passed one
that was stringified again, so the Station received JSON of JSON and
`request.json()` handed the route a `str`. `clearStaging` had the identical
defect. `reindex`, written a version earlier and sitting on the next line,
passes the object and always worked.
The server half is the part worth keeping. Twenty-one routes read
`await request.json()` and called `body.get(...)` without checking the
shape, so any non-object body — a bare string, a list, a number — raised
and was dressed by the `Exception` handler as E_INTERNAL/500, telling the
caller "this is a defect on the server" about a request they had malformed.
`_json_object()` now refuses it with E_SCHEMA naming what arrived. C.12
already required this; nothing had asked the question.
Reaching that refusal needed one more thing: the only registered handler
was `Exception`, so a VineError raised outside a route's own try/except was
re-labelled 500 with the wrong story. The code the refusal already carries
now decides the status.
Separately, the Explore console crashed on every branch in tree mode —
React #31, an object rendered as a child. `{d.coverage}` is C.2's
`{notes, branches}`. Every branch carries it and no leaf does, which is why
the mode whose entire job is branches could not open one, with or without a
node in the address. It renders the counts now, with a locale key in
pt/en/es.
`test_body_shape.py` covers both halves: the server refusing every
non-object shape on every body route, and a source check that no call site
in `api.js` stringifies a body the helper will stringify again — the same
trick `test_studio_calls` uses, for the same reason.
Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
claiming the helicopter lands on the file Two panels, one session, and both were saying something the engine does not. The dense share, in both consoles. v0.71 names `embed_ms` and `dense_ms`; this is the half that reads them. `TraceSteps` is shared by Ask and Playground, so subtracting both shares per row and listing each once at the tail fixes them together — the host names, the console nets, and `retrieval_ms` keeps meaning the whole span. A single primitive carries no trace (a one-step list would repeat what the header says), so the shares also ride `Server-Timing` as clocks INSIDE `vine`, never beside it: a consumer wanting the forest's own work subtracts, one summing every clock must not. A bare hybrid `locate` read as 109 ms of forest work; it reads 1.5 ms now, which is what it always was. The path panel. The line it drew climbed `parent` from every hit to the root, in one colour, and that is an ADDRESS — it exists on a sweep, which never moves at all. Animating it made the claim worse: footsteps down from the root, telling the reader the agent had walked there, which is precisely what the product does not claim. Now the flight is blue and STOPS at the branch (a segment whose destination is a marked node is not its to draw), the monkey's own movement is amber and means that in both modes — on a sweep the one step into the opened file, on a walk the real hop sequence from `hopSegments`, where only a hop that names ONE node counts as a position and a hop that returned a set is the agent looking around from where it already stands. Every leg advances on one clock rather than staggered by depth: several hits are several drops leaving the base at once, and revealing them in sequence invented an order the retrieval never had. Each node the trail touches is named, way-through captions dimmer than results, and the camera leans on what has been revealed so far so the view travels instead of framing the answer before it arrives. The panel's switch moves from an icon alone in the header into the controls row, with the icon before the label. J.5.15 rule 10 is kept by moving rather than by staying: the drawing is still not a parameter of the question and still absent from the flags list below, and that row sits above the answer so it is not pushed off screen — which was the reason the header was chosen. `Toggle` gains a compact variant for it: nesting a `<label>` around a `<button role="switch">` associates nothing and leaves half the target dead to the pointer, so the whole row IS the switch. The panel's test was rewritten twice as the design settled, each version asserting the current rule and failing on the previous one — verified both times by putting the old behaviour back. Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
Spec v0.72. The path panel changed in five ways over one session and the document describing it did not, so this is the amendment — written after the drawing settled rather than before it, because each version of the drawing refuted the one before and a spec cut in the middle would have been cut twice. Rule 5 said the trail is the forest's own structure, and it is: a chain up `parent` from each hit to the root. What it did not say is what a reader sees, which is a journey — and the journey it showed was the agent arriving in one move on the exact document it needed. That is the claim an entry search does not make. The value of dropping somebody NEAR is lost the moment the picture shows them landing ON. The split is mechanical rather than editorial, which is what stops the old claim returning by tuning: a segment whose destination is a marked node belongs to the walk, every segment above it is the flight, and the flight is therefore structurally incapable of entering a document. The walk is the agent's own movement in both modes — on a sweep the one step from the branch into the opened file, on a walk the real hop sequence, where a hop naming ONE node is a position and a hop returning a set is the agent looking around from where it already stands. Rule 6 gains the second colour token (a shortcut is a third fact and stays a third colour, and neither line may take a hue the dots use). Rule 9 gains the movement rule: both lines move, because a still line beside a moving one reads as a wire beside a route, and every leg advances on ONE clock — several hits are several drops leaving the base together, and sequencing them invents an order a ranked set never had. Rule 5 also gains the captions: every node the trail touches is named, way-through names quieter than results and yielding to them when the map is crowded. Rule 10 gains the camera, which leans on what has been revealed rather than on the final set: a view that frames the answer before it arrives has told the viewer there is nothing to discover. F.151-F.153, all three read off the source on F.137's stated boundary — the canvas cannot be inspected from a test — and each verified to fail with the previous behaviour put back rather than merely asserted to pass. Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
Caught by looking at the panel, which is the only way this one could have been caught: the rule was stated as a property of the DESTINATION and the sweep is the case where that happens to be right. "A segment whose destination is a marked node belongs to the walk" is true of a sweep, where the walk IS the last ancestry step. On a walk the walk is the hop sequence and covers no ancestry at all, so those segments were yielded by the flight to a line that never drew them. Drawn by nobody. The address vanished and the map became hop lines flying between distant branches with no forest under them — which is a worse picture than the one v0.72 set out to fix, since it removes the structure instead of misdescribing it. The rule is about COVERAGE: the flight draws every ancestry segment the walk does not. On a sweep it yields the last step and stops at the branch; on a walk it draws the chain whole, and the amber hop sequence is visibly a different journey over it. Spec, CLAUDE.md and F.151 all restated, and F.151 now asks the question that would have caught this — every segment drawn exactly once, by one line or the other and never by neither. Verified by putting the destination rule back: it fails. Worth naming because it is a shape, not an accident. A rule written from the case in front of you describes that case and constrains nothing else, and its failure mode here was silence: nothing raised, nothing logged, a line simply was not there. The canvas cannot be inspected from a test (F.137), so "drawn by nobody" is exactly the class of defect that reaches a person's eyes first — and this one did. The legend also drops the metaphor for the mechanism: `Fly` and `Walk`, untranslated, on J.5.1's precedent for `MCP` and `API`. The explanatory tail stays in the reader's language. A person watching two dashed lines does not need to be told there is a monkey. Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
The Audit console showed when, who, which call, `ok`, a byte count — and the When column was blank, because the field is `ts` and the console read `e.at`. Under it, three facts the row never carried at all. Cost: J.4 has said since v0.35 that a store hit is audited with "the cost avoided, never a second spend" — a sentence about a column that did not exist. Which refusal: `result` was ok or error, so somebody reaching for a scope they do not hold and a mistyped id were the same row. What it took: the engine times every primitive and the host serves the figure in a header, and none of it survived the request. The route took `limit` and `principal`, so every other filter belonged to whoever was reading, over whatever page they held — which makes every summary a summary of a page, a number that moves when somebody changes the limit. Now the filters are applied in SQL before the page is cut and the totals are computed over the whole filtered set. Spend and saving are separate sums off one column, split by `result`, so nothing bills a deployment for the calls it avoided. Spec v0.73: J.4.2, J.4.3, J.5.16. Acceptance F.154-F.157. Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
Part I opens with "A forest snapshot is ONE file", and that stopped being
true the day `--with-payloads` was added. A forest holding a dataset
needed a bundle AND a sidecar, separate at every step: two files written,
two links to click, two upload fields with the second one optional and
defaulting to off. Nothing tied them, so nothing stopped them coming
apart, and the half that goes missing is precisely the half git cannot
hold — that is what the sidecar was invented to carry.
It came apart in the field. A 1,877-node forest was imported from a bare
bundle; the audit row reads {"nodes": "1877", "payloads": "0"}, status ok.
Two dataset passports named a payload that was not on the volume, the
restore knew it — it had just rebuilt the catalog off those very passports
— and it said nothing. Found two days later, by a 404 in a console.
The mechanism was never broken. It created and restored a sidecar
correctly the whole time, which is why the fix is not in it. Excluding
payloads is legitimate: a metadata-only snapshot is a real artifact, and a
restore that refused to produce one would strand every bundle taken before
today. What is not legitimate is `payloads: 0` meaning both "this forest
has none" and "you did not ask", or a 200 that reports how many payloads
were packed while never reporting how many nodes are now dead. Both counts
were free — the producer walks the tree, the restore rebuilds the catalog.
Neither was taken.
So the artifact is <forest>-<stamp>.forest: a zip of forest.bundle,
payloads/<forest-relative path>, and a README carrying the two commands
(unzip, git clone forest.bundle) that open it with no MonkeyLLM present.
A container is exactly what can hide the anti-lock-in property, so it is
written inside the file rather than assumed. Payloads travel by default
everywhere; create reports payloads_omitted and restore reports
payloads_missing, the latter through catalog.count_missing_payloads shared
with C.17 rule 11, because two counts of one condition agree only where
somebody compared them. The shape is read from the file's content — bundle
magic at offset 0 first, is_zipfile second — never from its name, which on
the import route is a claim by whoever is importing.
The container closes a wider hole nobody had reported: `_assets/` is in
the forest's own .gitignore and was in no sidecar either, so a type: media
node survived no snapshot at all. It is per-branch (G.5.1), so it is a
directory name at any depth.
Member rules widen with it, and widening is where such a rule gets lost:
under payloads/, no `..`, no backslash, no component beginning with a dot,
and then a dataset database or something under `_assets/`. The dot clause
is load-bearing and my own negative control proved the tests did not cover
it — payloads/.git/config was already refused by the extension check, but
payloads/.git/_assets/x rides the `_assets` clause straight into the
repository git reads on the next commit. Both cases are in the suite now,
and it fails with the clause removed.
Spec v0.74: Part I, J.13.1, J.13.2. Acceptance F.158-F.161.
Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
Nine spec versions since 0.65.0 (v0.66 through v0.74). Behaviour changes worth reading before upgrading: - Snapshots are now one file, <forest>-<stamp>.forest, and payloads travel by DEFAULT. Any caller relying on the old default — the Station's with_payloads, the CLI flag, api.takeSnapshot — now moves the dataset and media bytes as well as the map, which for a large forest is a different amount of bandwidth and disk than it was yesterday. Opt out with --no-payloads or with_payloads: false. Snapshots taken before this release still restore, bundle and sidecar alike; the shape is read from the file's content, not its name. - The answer role's max_tokens default moved 600 -> 1500 (v0.63), applied to existing bindings once by a stamped data repair, which invalidates each forest's answer cache one time. - locate derives its search terms from the question (v0.70) instead of handing FTS5 the whole sentence: recall@1 0.711 -> 0.778 on the labelled set, and a call whose derivation is empty still searches the raw query, byte-identically. - The Audit route applies its filters in SQL before the page is cut, so the totals describe the filtered set rather than the page (v0.73). Signed-off-by: Jimmy Wesley <rededown@hotmail.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 0.74.0 — the instruments stop flattering the product
Nine spec versions, v0.66 through v0.74, and the package moves 0.65.0 → 0.74.0.
Six of the nine are the same defect wearing different clothes: the product was
reporting something other than what happened, and every time it was reporting
it in the product's own favour.
None of those were lies anybody wrote. Each was a true number in the wrong
account, or a true row with the load-bearing field missing.
What changed
Two clocks came out of the forest's bill (v0.68
embed_ms, v0.71dense_ms). With hybrid entry on,locate's span contained an HTTP roundtrip to the embedding provider and a linear scan over every node vector. Both
now ride the Part D event beside the engine's own figure. They stay apart on
purpose: whoever is dominated by
embed_msbuys a closer embedder, and whoeveris dominated by
dense_msneeds an index instead of a scan — one merged"hybrid overhead" figure sends half its readers to fix the wrong thing. Host
NAMES, console NETS:
elapsed_msandretrieval_mskeep meaning the whole spanto the byte, and
TraceStepssubtracts per row.v0.42's "the vector scan was never the problem (0.044 ms per dim-1024 dot
product)" was a true claim about one comparison. A hybrid entry makes one per
node. That sentence survives its own arithmetic only while the corpus is small,
and it is now quoted with its date.
A sample stopped being the corpus (v0.67). A 1,200-node forest asked what it
was about answered from the readme on a walk and from five ranked excerpts on a
sweep — both were the spec working as written, which is why the repair is four
spec sentences and not a patch.
coveragejoins the walk's whitelist; the sweepprompt must state that its material is a ranked top-k of something larger;
answergainsterms; and corpus-scope questions route to structure, because asingle document is one node's claim and never the corpus.
The trail panel stopped drawing a journey nobody took (v0.72). It drew one
line from the root to each hit, up the
parentchain — the forest's ownstructure, and a reader sees a JOURNEY: the agent arriving in one move on the
exact document it needed. That is the claim an entry search does not make; the
whole value of dropping somebody NEAR is lost the moment the picture shows them
landing ON. The split is about coverage: the flight draws every ancestry
segment the walk does not. Written the other way round it orphans those segments
on a walk — which shipped, and was caught by looking at it.
The audit log became readable (v0.73). Three facts the row never carried:
usd,tokens,calls,priced, off the provider's own usage and catalogueok/error— a scope probe and a typo were the same rowerror_code, the envelope's code and never its messagems(the Part D slice) andmodel_ms, apartThe route took
limitandprincipal, so every other filter belonged towhoever was reading, over whatever page they held — which makes every summary a
summary of a page, a number that moves when somebody changes the limit.
Filters are applied in SQL before the page is cut;
totalscovers the wholefiltered set. Spend and saving are separate sums off one column split by
result, so nothing bills a deployment for the calls it avoided. All new fieldsare nullable and omitted when absent: a pre-v0.73 row reads as absent, never
as zero, because
0.0 msand$0.00are both claims.A snapshot became one file (v0.74).
<forest>-<stamp>.forest: a zip offorest.bundle+payloads/…+ a README with the two commands that open itwithout MonkeyLLM. Payloads default ON;
createreportspayloads_omittedand
restorereportspayloads_missing. The trigger was a real forest importedfrom a bare bundle —
{"nodes": "1877", "payloads": "0"}, statusok, twodataset passports pointing at bytes that never came, found two days later by a
404 in the Data console.
The three that are not that shape
Live hops (v0.66).
answertakes an optionalrun;GET .../answer/{run}/eventsstreamsretrieval, onehopper completed step, anddone. The safety rule is deliberately not J.16's: a webhook leaves theStation's authority behind, while this is pulled by the same principal under the
same credential — so an event may carry anything the completed response would
have carried to that principal and nothing more. Emission never blocks, nothing
outlives its call, and the stream is never the answer.
A path is not a syscall (v0.69).
Forest.path_fordecided containment withPath.resolve()— arealpathwalk, one syscall per component — once per nodein scope. The split is by provenance, never by cost: an id arriving from
outside the engine resolves symlinks, an id the engine read out of its own
catalog does not. The failure mode is silent, so
tests/test_containment.pyisa surfaces × escapes matrix and a surface absent from it is not covered.
The question is not the query (v0.70).
locatehanded the whole sentence toFTS5, so every article and preposition was a search term with a vote. It now
calls
harvest's ownderive_terms— never a second derivation. Theload-bearing half is the fallback:
api,sql,uiderive to nothing, so anempty derivation searches the raw query and those calls stay byte-identical.
Measured
locatenet, cold / warmsniff(1,877 nodes)locaterecall@1locateMRRlocatep50The ranking numbers come from
bench/questions-locate-v1.json(T17), which isitself part of this release: 60 questions over a 1,877-node corpus,
expected_nodeswritten from reading documents and never from running asearch. It exists because two of the three previous labelled sets scored
recall@1 = 1.000 before any change — a ranking change measured on a saturated
set has not been measured.
Known gaps, stated rather than discovered
takeSnapshot(forest)now sends payloads by default (wasfalse). Thisis the fix for the 0-payload import above, and it is a behaviour change for
any existing caller relying on the old default — expect larger downloads.
Acceptance:line.F.137itself is present in the document body.subscriptions/listenend-to-end is not in the suite: the response never endsand the in-process client cannot hold one open without holding its own
shutdown (F.135 records the gap).
locatereturns results for all 15 questions the forest cannot answer, andstrengthis normalised against the best hit in its own result set, so thetop result scores 1.000 for all 60. That is C.1.1 from the other side and
wants its own round.
Acceptance
F.138 – F.161. Full suite green.