Repository navigation
feat(compute): MVP-B CPU inference worker (draft — blocked on the frozen output budget) - #153
Draft
gkalombo21 wants to merge 1 commit into
Draft
gkalombo21 wants to merge 1 commit into
gkalombo21 wants to merge 1 commit into
Conversation
…d on the frozen output budget) Adds the real CPU inference worker for the frozen kaayu-cv-extract:v1 profile: a long-lived llama-server (llama.cpp build b11669 = release v0.7.0 @ 620d2225, CPU-only, loopback, GPU layers 0) verified at start-up by the pinned executable SHA-256, its self-reported build and commit, /props, and the model file's size and SHA-256 (3605803b...e597); the frozen profile's prompt, schema and job-spec digests are checked before readiness. ExecutionProfile gains three default methods (spec_tags, accepts, execute_task) that reproduce the reference behaviour; the worker calls them. ProfileDispatch runs [reverse-bytes, mbongo-infer:v1] with no fallback. Every task goes through the MVP-A preflight and finish; the request carries only job-spec parameters and the pinned schema as a response_format grammar. A bounded loopback HTTP client, restart of a dead or hung server, safe observations (no CV, prompt or result in any log). scripts/setup-mvp-inference-runtime.sh acquires and verifies both artifacts outside the repository and quarantines on mismatch. mvp_inference_harness runs the real corpus and a live-chain end to end. CI group cpu_inference_worker (31 tests, scripted backend, fake HTTP server). Real results on an i7-8850H: 8/10 synthetic CVs schema-valid; the other two stopped at the contract's max_tokens = 400 (finish_reason "length") and were refused, as specified. The frozen output schema permits ~2,000 tokens of JSON; resolving that needs kaayu-cv-extract v2 under MVP_v0.1 §20. Live-chain end to end PASS with an in-flight server kill and recovery. No ComputeTask, Receipt, RPC, SDK or MVP-A change. Refs #148 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 of 5 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MVP-B — Real CPU Inference Worker (DRAFT: blocked on the frozen output budget)
Tracks #152 (child of #148). Baseline
20d46774bdb1890b3f510e0ed8e31552dd41c96a. Not for merge untilkaayu-cv-extractv2 resolves the output-budget defect described below and the real corpus reaches 10/10.What it adds
crates/mbongo-compute/src/cpu_inference/: pinned runtime configuration (llama.cpp buildb11669= releasev0.7.0@620d2225, Windows and Linux CPU archives andllama-serverSHA-256), start-up verification (frozen profile digests; executable SHA-256;--versionbuild and commit; model size and SHA-256;/propsbuild and model), a bounded loopback HTTP/1.1 client, the long-livedllama-servermanager (structured args,--n-gpu-layers 0,--parallel 1,--no-slots,--no-webui,--no-cache-prompt, restart of a dead or hung server), the backend trait with a scripted test double,InferenceProfile(MVP-Apreflight→ frozen two-message prompt → one completion with job-spec parameters and the pinned schema asresponse_format→ MVP-Afinish) andProfileDispatch(no fallback).ExecutionProfilegains three default methods (spec_tags,accepts,execute_task) whose defaults are the reference behaviour;worker.rschanges only its three call sites.scripts/setup-mvp-inference-runtime.sh: acquire and verify both artifacts outside the repository; quarantine and exit non-zero on mismatch.mvp_inference_harness corpus|e2e: real runs (never CI).test-vectors/inference/synthetic-cv-corpus-v1.json: 10 invented CVs + 3 must-refuse inputs.cpu_inference_worker(31 tests) as step Mbongo MVP CPU Inference Worker; INDEX row;docs/product/MVP_CPU_INFERENCE_WORKER.md.Real evidence (i7-8850H, 6 threads, CPU only): runtime and model verified exactly (model SHA-256
3605803b…e597, 2,497,281,120 B). Corpus 8/10 schema-valid; warm P50 48.6 s / P95 55.4 s; cold start 3.5–3.8 s; peak working set 6.3 GB; 3/3 invalid inputs refused before the model; in-flight kill →INFERENCE_FAILED→ restart → success. Live chain end to end PASS (task accepted, killed first attempt, retry, receipt anchored and verified, commitment matched, chain and log scans clean).The blocker: R08 and R10 hit
max_tokens = 400(finish_reason: "length") and were refused asOUTPUT_NOT_JSON; with room they complete (463 and 1,331 tokens). The frozen schema allows ~2,000 tokens of JSON. Fixing it changes the frozen profile — a decision for change control, options in #152 and the doc §16.4.Unchanged: 76 protected blobs (RFC 0005, RFC 0002, RECEIPT_SPEC, lock v0.4, rpc_v0.3, consensus, SDK, conformance, J, K, SNP adapter, protocol vectors,
MVP_v0.1.md, the MVP-A contract, module, profile files, vectors and test group) identical to the baseline.Refs #152, #148
🤖 Generated with Claude Code