Repository navigation
Add optional persistent task bridge for personal Dot MCP Events - #4
Merged
Merged
Conversation
Record approved ChatGPT-only MCP Events queue, separately paired local clients, durable idempotency, and accepted-job no-fallback requirements. Text routing and background jobs are distinct; live Dot roundtrip is required before readiness. Documentation only; source/design review completed, baseline go test ./... passed in sandbox.
Plan queue, authenticated event/client surface, runtime integration, Magpie capability and accepted-job barriers, provider/background tasks, then real Dot probe. Existing baseline passed; implementation verification remains per-task. Approved execution continues without another confirmation step.
Implement internal/dotqueue Store API with AES-GCM persistence, flock serialization, idempotent submit/claim/complete, cancel intent, explicit Recover, and bounded validation. Tests: go test ./internal/dotqueue/ -count=1 (pass); run_checks dotqueue-race -race (pass); run_checks dotcontrol-full go test ./... (pass).
Address round1 I1-I7 and M1: validate worker capabilities, reconcile unknown outcomes, gate sensitive reads, bound all UTF-8 content, strictly validate authenticated state, enforce marker/privacy checks and lock the actual validated handle. Persist progress summaries and authenticate envelope headers. Behavioral RED retained: dotqueue-round1-red (exit 1 before fixes), cancel-red and url-red (exit 1). Final GREEN: isolated runner --race go test ./internal/dotqueue -race -count=1 -v, exit 0; dotqueue-round1-final-race.log/.json. Includes actual child-process submit/claim CAS and Windows ACL/junction fixtures. Initial race filesystem failure diagnosed as MoveFileEx access denied; bounded replacement retry added. No full-suite rerun; independent review pending.
Round2 residual I4: distinguish WHATWG numeric final labels from DNS names without integer overflow or DNS lookups. Reject hex/octal/shorthand/integer aliases with optional trailing dot, IPv4-mapped IPv6 and scoped addresses; retain canonical public IPs and ordinary DNS hosts. Add literal rejected/allowed URL table before changing validation. Behavioral RED: isolated run_checks.py dotqueue-round2-red go test ./internal/dotqueue -run ^TestArtifactHostNumericAliases$ -count=1 -v; exit 1, compiled assertions reproduce hex loopback/private bypasses. GREEN: dotqueue-round2-green go test ./internal/dotqueue -count=1 -v exit 0 (3.43s). RACE: runner --race dotqueue-round2-race go test ./internal/dotqueue -race -count=1 -v exit 0 (13.31s). Logs and JSON retained in magpie-context-verification. No full-suite rerun or DNS safety claim; I4 pending independent review.
Implement independent encrypted subscription state, generation-checked JSON callback verification, official event occurrence fields, Standard Webhooks exact-byte signing and secret rotation. Pin each connection to validated public DNS addresses with normal TLS verification and no proxies or redirects. Reserve per-subscription delivery attempts atomically and persist bounded retry and acknowledgement state. Evidence: behavior-first scaffold test failed on invalid subscriptions and reserved IP acceptance; official-wire regression failed on discovery and JSON challenge verification; reserved-address and quoted-TTL regressions failed before fixes. Final go test ./internal/dotevents -count=1 passed (1.811s); go test -race ./internal/dotevents -count=1 passed (4.197s, CGO/MinGW); go vet ./internal/dotevents and git diff --cached --check passed. All callbacks use local TLS fixtures; production dial guard tested independently with offline resolver/dial functions. Full suite deferred to integration controller.
…oundary Use source event ID in both webhook-id and eventId; preserve caller occurrence time across delayed and reopened retries. Start new stores disabled. Cache verification for five minutes by authenticated principal and URL, invalidate on revocation, and preserve accepted subscriptions without TTL extension on generation-safe refresh failure. Validate canonical local non-root paths and every ancestor on every transaction; reject Windows device/UNC/ADS/reserved paths and linked ancestors; protect directory and owned files on reads and writes; validate bounded marker contents. Add independent HTTP identity/timestamp assertions, complete-fixture strict mutation tests with zero-I/O counters, cache/revocation tests, Windows junction/DACL and Unix permission/link tests. RED: go test ./internal/dotevents -run TestReview -count=1 failed on header/body identity, refresh challenge count, default-enabled new store, invalid marker and noncanonical paths. GREEN: focused tests PASS 2.582s; race PASS 4.788s with CGO/MinGW; vet PASS; Linux amd64 test binary cross-compile PASS (not executed); git diff --check PASS. No live external network, new dependencies, or runtime integration.
Carry OccurredAt through queue notices into event delivery, avoid recover deadlock, and add fixture-backed subscription verification for readiness tests. Expose dotevents.NewDelivery for offline fixtures and move MCP bridge tool test out of dotbridge to break import cycles. Verified: go test -run TestBridgeAPIProbeClaimComplete|TestNewWithBridgeAddsTaskTools ./internal/dotbridge ./internal/mcpserver
Fix C1/C2 and I1-I6: preserve implicit HTTP 200, narrowly patch modern SDK discovery, bind the trusted consumer to paired queues and claims, recheck gates after I/O, synchronize enabled reads, preserve concurrent revoke, repair cancel routing and degrade optional bridge failures. Flatten Events params and bound readiness to subscription generation/expiry; hash nonce expectation and refuse unverified task mode. Behavioral RED: reproduced real SDK HTTP panic/EOF, cancel 404, stale revoke, post-I/O pause and valid JSON rejection. GREEN: targeted dotbridge/mcpserver/dotqueue/dotevents tests; CGO MinGW race dotbridge/mcpserver; optional runtime vault test; go build ./...; targeted go vet. Full fixture covers authenticated MCP HTTP, TLS callback verification/notice, generated paired queue, probe completion/readiness and client answer result read.
Preserve callback verification failures across competing refreshes, while returning disabled/stale for paused or revoked state. Check persisted cache deadlines with legacy zero compatibility and bounded verification allowance. Protect newly created intermediate ancestors without changing existing parents. Compare failed-refresh TTL with the most recent successful refresh exactly. Keep NewDelivery and production public HTTPS transport unchanged. Evidence retained under: C:\Users\13678\AppData\Local\Temp\opencode\magpie-context-verification - dot-events-minor-red.json/.log: TestMinor reproduced error masking, invalid persisted cache acceptance, and inherited intermediate ACLs. - dot-events-minor-race.json/.log: go test -race ./internal/dotevents -count=1 PASS (package 5.367s, runner 8.32s). - dot-events-minor-vet.json/.log: go vet ./internal/dotevents PASS. - gofmt and git diff --cached --check clean. External report line 11 corrected and evidence appended. No full-suite run; controller owns integration checks. Commit contains only owned events files.
- Offline bridge enable writes dot_bridge_enabled via config.Save and requires restart before admin enable - Pair admin output redacted; probe --mode answer|task; jobs uses GET /dot-bridge/v1/jobs - Private pairing/state file helpers with ACL, link checks, and atomic replace - Document operator flow and controller integration gaps in research report
Accept the explicit answer/task mode emitted by bridge probe, retaining answer as the default for older callers. Forward to StartProbeMode so the authorized admin path can create the independent background capability probe. Focused operator and HTTP mode tests passed via the isolated runner; live Dot acceptance remains outstanding.
Keep the existing retention regression aligned with the dedicated capacity error while leaving changed-request conflicts intact. Full internal/dotqueue race suite passed via isolated runner: 30.91 seconds; retained dotqueue-capacity-full-race.log.
Accept authentic legacy verification timestamps without guessing a bound for lock/save/HTTP lifecycle time or backward wall-clock movement. Keep structural timestamp and verification checks on load. Bound every cache hit by the prior proof, source grant, new grant and now plus a shared five-minute cache TTL. Clamp fresh verification to subscription expiry and keep refresh CAS/revocation and public HTTPS protections intact. Deterministic tests cover legacy 10s/15s/25s spans, backward clocks, shorter TTL, source expiry, repeated refresh non-extension, reopen and invalid schema. Evidence: C:\Users\13678\AppData\Local\Temp\opencode\magpie-context-verification - dot-events-minor-clock-red.json/.log: behavioral failures reproduced. - dot-events-minor-clock-race.json/.log: go test -race ./internal/dotevents -count=1 PASS; package 6.188s, runner 9.49s. - dot-events-minor-clock-vet.json/.log: go vet ./internal/dotevents PASS. - gofmt and staged diff check clean. No full suite or subagents. Append rationale/evidence to external dot-events-task2-report.md. Correct the explicitly requested stale GCC statement in the List report using retained controller race evidence; no queue source/test changes.
Address scoped review test gaps by comparing the complete persisted per-mode proof after completion replay and validating HTTP result identity, terminal state, text and artifact URL/label. Focused race checks passed in 6.71 seconds; retained dot-proof-review-assertions.log. No live Dot execution claimed.
Replace staging-only task descriptions with the implemented mode-specific nonce probe, one-time proof consumption, bounded expiry, and durable background result workflow. Retain the distinction between authenticated workflow proof and actual account/computer acceptance.
Explain that Magpie imports into the OS keyring while DotControl task commands still read the protected pairing file. Avoid recommending deletion followed by commands that depend on that file; distinguish file removal from credential revocation.
Close the capacity coverage gap recorded in the dot-queue capacity review: no test exercised two distinct new requests competing for the single free slot of a store already holding 99 jobs. Two real child OS processes submit distinct request IDs against a store retaining maxRetainedJobs-1 jobs and are released together by an inherited pipe byte after both publish a ready file, so the winner is decided by the queue lock rather than by a timed sleep. The test asserts exactly one success and exactly one queue_full, a retained count of 100 after reopen with an unchanged CapacityStatus, the winner present and idempotent at capacity, the loser absent, and no plaintext prompt, request, client or job id in the encrypted state file. Fault injection through `go test -overlay` (no repository file modified) confirmed the test fails when the capacity guard is removed, made off-by-one, made unreachable, moved ahead of the idempotency lookup, or when the index, order, ByRequest lookup or returned public job is corrupted. Focused and package verification on go1.27.0 windows/amd64 with the MinGW gcc from C:\msys64\mingw64\bin on PATH: - go vet ./internal/dotqueue/ exit 0 - go test ./internal/dotqueue/ -run ^TestCapacityLastSlotCrossProcess$ -count=1 -v PASS 0.52s - go test ./internal/dotqueue/ -run ^TestCapacityLastSlotCrossProcess$ -count=3 -v PASS 3/3 - go test ./internal/dotqueue/ -run ^TestCapacityLastSlotCrossProcess$ -race -count=1 -v PASS 6.72s - go test ./internal/dotqueue/ -run ^TestCapacityLastSlotCrossProcess$ -race -count=5 ok 41.031s - go test ./internal/dotqueue/ -count=1 ok 6.440s - go test ./internal/dotqueue/ -race -count=1 ok 47.376s Test-only change; no production file and no other test file modified.
Both reported concurrency tests reproduced with unknown-error stacks at saveState replacement. Loaded diagnostics captured MoveFileEx access denied; concurrent directory ACL propagation alone reproduced the same syscall denial with protected source and target ACLs in all three isolation repetitions. Match dotqueue's bounded one-second retry for access-denied and sharing- violation errors under the existing store lock. Add a real Windows handle regression that fails before the fix and passes after reader release, plus missing-source preservation coverage. Retain evidence and limits in the concurrency diagnosis report; remove all temporary instrumentation. Validation: Windows Go 1.27 MinGW race regression x20 PASS (3.356s), entire events package race x30 PASS (159.376s), vet and diff checks PASS. Full-suite race passes events (7.580s) but fails outside scope in cmd/dotcontrol: TestUninstallWaitsForProcessAfterEngineLockRelease context deadline exceeded.
Use atomic pause/revoke flags in capacity status fixtures. Capture event-store child stdout/stderr and wait for every child before reporting failures, preserving useful diagnostics under full-suite load. Capacity fixtures passed three race iterations in 26.67s; event cross-process diagnostics passed twenty race iterations before the separately committed Windows replacement fix. Latest full-suite uninstall timeout remains under investigation.
Remove post-Open callback replacement from pause/revoke fixtures. Configure immutable lifecycle callbacks before the service starts its worker, retaining atomic flag transitions for test interleavings. The observed callback-pointer DATA RACE is addressed across all matching fixtures. Focused write-after-pause race count20 passed; complete repository go test -race -count=1 -timeout 3m ./... passed in 59.7 seconds, retained dotcontrol-startup-gates-final-race.log.
events/list now returns delivery, inputSchema and payloadSchema from one shared catalog; disabled or unauthorized principals receive an empty array instead of null. events/subscribe returns the flat {id, refreshBefore, cursor, truncated} result required by the MCP Events wire instead of a nested subscription object. Verified with real HTTP handler tests and race runs over mcpserver, dotbridge and dotevents (39.93s).
events/list gated on per-queue authorization, so the authenticated MCP consumer received an empty catalog and could never subscribe. Discovery now requires only an enabled bridge; subscribe still enforces queue authorization and revocation. Verified by handler and race tests over mcpserver, dotbridge and dotevents (41.52s).
…ence Map verification failures to JSON-RPC -32015 with classified data.reason (timeout, challenge_failed, url_rejected) so the platform can report specific causes. Align event occurrence body with the official eventId/name/timestamp/data schema. Verified with classified failure regression tests.
Use jsonrpc.Error so callback failures carry real -32015 codes and data.reason. Distinguish network, timeout and URL policy rejection; retain allowlisted encrypted queue-scoped diagnostics across restart and suppress them after authorization changes. Expose only a stable bridge open failure code. Regression tests exercised prior failures; full repository race run passed in 68.12s, vet and diff checks passed. Actual platform subscription root cause remains unverified until deployed diagnostics observe an attempt.
Persist only allowlisted URL rejection rule names in queue-scoped diagnostics, preserving existing URL/public-address policy and legacy records. Subscription/status regression tests and related package race checks passed (41.81s); scoped source review approved. Actual callback rejection rule remains pending a new attempt.
Networks running fake-ip DNS (Clash, sing-box, Surge) answer every name with a synthetic address, so the public-address guard rejected the callback before any connection was attempted. Production diagnostics confirmed the path: the diagnostic carried reason=callback, stage=callback_url, rule="". The empty rule proved the rejection came from the dial path rather than URL pre-validation, and the encrypted state held zero subscription records. Resolve callback hostnames over DNS-over-HTTPS instead. The SSRF guard is unchanged: every candidate must still pass publicAddrs, the CNAME chain must begin at the queried host so a provider cannot substitute an unrelated hostname, and the system resolver remains the fallback. Implemented with the standard library only, because the Go module proxy is unreachable from this host. Classify failures honestly. A hijacked resolver is a network condition, so it now reports network rather than url_rejected, which had been sending operators to look for a URL problem that did not exist. publicTransport preserves the resolver's own error code instead of flattening it to callback. Providers are queried concurrently under one shared budget, and verified answers are memoized on the Delivery so a burst of deliveries and retries shares one resolution. Verified live on this host: chatgpt.com, openai.com and multi-hop CNAME hosts now resolve to public addresses. Full package race run passed in 51.55s. The cmd/dotcontrol legacy-reference failure is pre-existing on clean HEAD c7055e4 and unrelated.
… renewals Persist subscription trust epochs separately from verification attempts and retain mode proofs for continuous renewals of the same authenticated callback binding. Tie readiness to the committed active lease instead of requiring a new manual nonce every ten minutes. Changed callback, signing secret, authorization, unsubscribe or an expired lease breaks continuity; legacy proofs do not gain sustainable trust without binding evidence. Expose sanitized renewal timestamps and failures while retaining valid old grants. ChatGPT remains responsible for refreshing before refreshBefore; DotControl never renews its own authorization. Existing accepted-job queries remain independent from readiness. Sample grant time under the state lock and recheck cached finalization so lock waits cannot preserve an expired epoch. Honor the minimum committed bridge and events lease after partial persistence. Deterministic regressions failed before the fixes and passed afterward; full repository race tests passed in 73.73s. Independent review closed both lease-boundary findings with no new important findings. Live migration and renewal acceptance are pending joint deployment.
GitHub CI failed on Windows and macOS while the same packages passed locally, because two fixtures assumed a particular platform environment. Windows: the runner's user is the built-in administrator (RID 500), whose SID Windows renders as the "LA" alias in SDDL text, so an ACL that grants only the current user failed a text comparison. Compare the ACE principal as a SID instead, matching the existing dotqueue check, and keep the protected-DACL, single-entry and full-control assertions. macOS: temporary roots sit under the system /var symlink, which the events and queue stores deliberately reject as an ancestor. Resolve the fixture root with EvalSymlinks, as codexbridge and the installer tests already do. Production symlink rejection is unchanged. Full repository race run passed in 76.61s.
VeraPyuyi
marked this pull request as ready for review
October 22, 2026 05:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DotControl currently exposes desktop and history operations but has no persistent task inbox that a personal Dot connected through ChatGPT can subscribe to. This change adds an optional task bridge: a separately paired loopback client submits a durable request, MCP Events notifies the configured consumer, and authenticated claim/progress/completion tools return results to the same job.
The bridge supports answer and background modes with independent nonce probes. Proofs bind to the active subscription and expire within ten minutes; completion retries cannot renew them. Queued cancellation is terminal, while cancellation after claim records intent. Restarted claimed work becomes outcome unknown and is never automatically reassigned.
Storage encrypts queue payloads and subscription secrets, uses private files and cross-process locking, and retains stable request identities. Pairing grants queue permissions separately from existing MCP/admin credentials. Outbound callbacks require public HTTPS, verification and Standard Webhooks signatures. The regular 13-tool MCP catalog remains available when the optional bridge cannot open.
Validation: the complete repository passed
go test -race -count=1 -timeout 3m ./...atc690372(59.31 seconds). The Windows command binary also builds. HTTP fixtures exercise subscription/claim/completion, independent answer/task proof, results/artifacts and lifecycle gates. Scoped reviews found no remaining implementation blockers in the queue, proof core or event cache; API review requested stronger HTTP budget and write-then-pause tests;e437da6adds real ready-client HTTP acceptance at exactly 2 MiB, overflow with no queue write, worst-case escaping, actual persistence followed by pause and original-ID recovery, and failed-list privacy/gate precedence. The full bridge package race run passed (15.710 seconds). This was additional coverage, with no production defect or production file change. The scoped test review closed both Important coverage gaps with no remaining Critical or Important finding. The pause case checks same-ID repeated submission idempotency; the separate actual-code interoperability run checks ByRequest GET recovery. Subsequent full-repository race repetition was not needed for these test-only changes.An offline interoperability run used actual DotControl producer code at
c690372and actual Magpie client code at8eb9d7d, with pinned-source hashes and Go overlays. It passed pairing, authenticated MCP HTTP subscription and nonce completion, Enqueue/ByRequest/idempotent repeat/Get/Cancel, a 262,144-byte escaped request (1,572,992 encoded bytes), the full escaped result, and artifact retrieval. Only fake vault credentials and loopback/TLS fixtures were used; no account, tunnel or live credential store was contacted. This is not a full-suite or race claim for the harness.Final Windows validation: the complete repository passed
go test -race -count=1 -timeout 3m ./...with the changes committed in9db5f33(59.7 seconds). Earlier failures were diagnosed and corrected rather than waived:9705d8fretries only transient Windows state-replacement denials;34ba3c5rejects impossible process ancestry caused by reused Windows parent PIDs, with conservative recovery behavior unchanged;9db5f33installs fixture callbacks before worker startup to eliminate the separately observed test DATA RACE. Both production fixes passed independent review with no Critical or Important findings. Events passed 30 package race iterations, and the process-tree regressions passed baseline comparisons and loaded repetitions. These are Windows results; Unix runtime validation remains outstanding.This is a draft. An authenticated ChatGPT read-only
devicescheck returnedNOT_FOUNDbecause the installed plugin's tunnel client had not been seen for 300 seconds. New task lifecycle tools were not visible in that session. Real answer and workspace task probes have not been observed. The queue retains at most 100 jobs, including probes and completed work; automatic archive/deletion is not implemented. No merge or release is requested until remaining review and account acceptance evidence is attached.