Conversation
…s probe
`ensureBrokerSession()` probes the persisted broker with a 150 ms budget:
return await waitForBrokerEndpoint(endpoint, 150);
A broker that is serving a turn keeps its event loop busy, so it can easily miss
that window. The probe failure is then treated as "the broker is gone" and the
session is torn down — which, once `killProcess` is wired up, kills a perfectly
healthy broker together with the app-server and the in-flight turn it owns.
Two changes:
1. Raise the readiness probe budget to 3 s, and never tear down a broker whose
process is still alive. A failed probe only proves the broker did not answer
in time; only `ESRCH` from `process.kill(pid, 0)` proves it is gone.
2. Serialize the check-then-create window with a stale-aware lock file.
Concurrent clients could all observe `loadBrokerSession() === null`, each
spawn a broker, and let the last writer win — orphaning the other brokers
along with their app-servers and in-flight turns.
Both new tests fail on main with the relevant assertion and pass with the fix.
Full suite: 93 passing (91 before + 2 new).
ApexAiOfficial
added a commit
to ApexAiOfficial/codex-plugin-cc
that referenced
this pull request
Sep 25, 2026
Audited against open upstream reports on openai/codex-plugin-cc; each defect was reproduced or verified against this fork and has a regression test in tests/substrate.test.mjs (dedicated fake app-server). - openai#302 unbounded waits: every RPC has a bounded wall-clock timeout (120s default, command/exec aware, CODEX_COMPANION_RPC_TIMEOUT_MS, 0 disables), and requests on a dead connection fail immediately. The turn watchdog does not use a silence timeout (legitimate turns are silent for long periods): it fails fast when the connection closes, and after quiet periods asks the app-server via thread/read whether the thread is still active, recovering a turn whose completion event was lost and failing only an app-server that stops answering. - openai#453 zombie broker: the broker exits when its app-server child dies, so the next caller starts a healthy broker instead of wedging. - openai#706/openai#707 retained subscriptions: the broker tracks per-socket thread ownership (subagents inherit their parent's owners) and sends thread/unsubscribe when the last owner disconnects; requests for a thread wait (bounded) for its in-flight unsubscribe. Simpler than openai#707 because this broker already serializes active requests and streams. - openai#762 + openai#768 together: broker acquisition is serialized; a broker whose process is alive is never torn down or killed because it missed a probe (the caller uses a private app-server); metadata is cleared only when the process is provably gone; only a just-spawned broker that never became ready is killed, identity-checked. SessionEnd's kill is identity-checked. - openai#574 RC3: non-retryable errors fail a turn even when completion is inferred. The upstream fix (any error fails) is wrong: error notifications carry willRetry, and retried turns can succeed. - openai#775: fileChange start events without a change list no longer throw. - openai#740 reproduced with real Codex 0.144.1: live thread/resume ignores the requested sandbox in both directions (write-capable threads stayed writable after a read-only resume), and a per-turn override persists to later turns. Every turn now sends an explicit sandboxPolicy. Also fixes a same-process withFileLockAsync bug (a second async holder treated the first as stale) and makes the test harness stop brokers when a test process is killed by a signal. Validation: npm test 156/156, tsc clean, no stray processes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ensureBrokerSession()probes the persisted broker with a 150 ms budget:A broker that is currently serving a turn keeps its event loop busy, so it can easily
fail to accept the probe connection inside that window. The failed probe is then treated
as "the broker is gone" and the session is torn down.
Today that teardown mostly orphans the broker (this is #753). But once
killProcessiswired up — which is exactly what #762 does — the same path kills a perfectly healthy
broker, taking down its app-server and the in-flight turn with it. The caller sees:
A second, independent race makes it easier to hit:
ensureBrokerSession()has no mutualexclusion, so concurrent clients can all observe
loadBrokerSession() === null, each spawna broker, and let the last writer win — orphaning the other brokers along with the
app-servers and turns they own.
Reproduction
Submit two background tasks in the same workspace a few seconds apart:
failed—codex app-server exited before the turn completed.completedrunning→completedcompletedFix
is still alive. A failed probe only proves the broker did not answer in time; only
ESRCHfromprocess.kill(pid, 0)proves it is gone (EPERMand PID reuse aretreated conservatively as alive).
callers share one broker instead of racing to create several.
If the lock cannot be acquired within its budget the code falls through to the previous
behaviour rather than failing the caller outright.
Tests
tests/broker-lifecycle.test.mjsadds two cases. Both fail onmainwith therelevant assertion and pass with this change:
a live broker must never be killed after a failed probeconcurrent callers must share one brokerFull suite: 93 passing (91 before + 2 new), no regressions.
Relationship to existing work
killProcessfor that teardown. The two are complementary, but mergingfix: terminate broker process when ensureBrokerSession tears down (fixes #753) #762 without this change converts an orphaned-broker leak into an actively killed
in-flight turn.
same class of failure.