fix(funding): wait for message pool readiness for all funded accounts - #207
Merged
Merged
Conversation
`forge create` intermittently failed while broadcasting the deployment: failed to look up actor state nonce: resolution lookup failed (t410fa34eb7lgf6xovtgfnsnoe6sz67ca2zjkbwi22ly): actor not found The funding step sleeps for a fixed interval and then verifies the transfers with `lotus wallet balance`, which reads the chain head. The message pool tracks its own tipset (`mp.curTs`), updated through a coalescer with a 2-6s delay (HeadChangeCoalesceMinDelay/MaxDelay), so it can still trail the head while the balance is already visible. `eth_sendRawTransaction` resolves the sender's nonce against that lagging view, and fails. Deploy Multicall3 and Deploy MockUSDFC share a parallel epoch, but MockUSDFC spends ~27s installing dependencies before it broadcasts, so only Multicall3 lands inside the window — it compiles in ~130ms and broadcasts immediately. Probe with `cast nonce --block pending`, the only query that goes through the same `getStateNonce(ctx, addr, mp.curTs)` call as the broadcast; every other block parameter silently returns 0 for a missing actor. The probe is read-only so it is retried; the deployment itself still runs exactly once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
Good find but I think it needs to be broadened. Every account funded at epoch 4 has this same problem and it's just multicall3 that's the first to need the funds. So instead, it seems we should move this check into |
rvagg
approved these changes
Sep 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cast nonce --block pendingbeforeforge createRationale
Deploy Multicall3 Contractintermittently fails during broadcast:Funding verification uses
lotus wallet balance, which reads the chain head, while themessage pool's tipset (
mp.curTs) can lag by the coalescer's 2-6s delay.--block pendingis essential: it alone routes throughMpoolGetNonceto the samegetStateNonce(ctx, addr, mp.curTs)lookup used by broadcast. Every other blockparameter returns
0, nilfor a missing actor, silently hiding the race.The probe is retried because it is a read-only query;
forge createstays outside theretry, as
src/utils/retry.rsrequires these helpers never wrap transactions ordeployments.
Both deployments start in the same parallel epoch, but MockUSDFC spends ~7s installing
dependencies and compiling before it broadcasts, which clears the coalescer window.
Multicall3 compiles in ~130ms and broadcasts immediately. In nightly run 35198060438 the
broadcast failed 0.6s after funding verification had reported
DEPLOYER_MULTICALL3: 10 FILfor that same account: the balance was already visible on the head while the pool still
could not resolve the actor.
Confirmed in 24 jobs across 20 of the 100 nightly runs between June 27 and September 17,
hitting one or two matrix legs at a time. Earlier occurrences are likely but no longer
verifiable, as Actions logs from before late June have expired. No issues were ever
filed for it, because
Start clusterusescontinue-on-error: trueand the step thatmarks the job failed runs after issue reporting has already decided not to file.
Verification
cargo fmt --all -- --checkcargo clippy --all-targets --all-features -- -D warningscargo test --all-targets --all-features(50 passed)Not verified end to end against a live devnet. On a clean run the log will show
✓ Message pool resolves the nonce for 0x06f840fd662faeeaccc56c9ae27a59f7c40d652a. Ifthe race is hit,
Multicall3 deployer nonce lookup failed (attempt 1/6), retrying in 4 seconds...appears first; the deployment proceeds only once a probe succeeds, and sixconsecutive failures abort the step instead.
🤖 Generated with Claude Code