Skip to content

Startup seeding resets persisted accounts after leader restart #195

Closed

Description

@snawaz

Problem

  • On restart, Keeper opens the saved AccountsDB and then unconditionally writes every account supplied in KeeperBuilder.accounts. The validator supplies MagicContext with zeroed data, so this replaces its persisted intent counter and can reuse intent IDs.
  • That startup write is not a replicated transaction. Verifiers keep the account state they replayed, while the restarted leader has the reset value. The next authenticated state seal detects the difference and stops the verifiers.
  • This regresses the V1 validator's behavior: V1 creates MagicContext only when missing and otherwise reads and retains its existing data.

Reproduction

  • Seed an account through KeeperBuilder.accounts, change its stored value, close Keeper, then reopen the same database with the same builder account. Before the fix, the startup value overwrites the saved value.
  • In the validator, redhat/replication_settlement_recovery schedules settlement, restarts the leader, and checks the next state seal. In the failing CI run, both verifiers exit with SealMismatch(2): the leader and verifiers each record 17 transactions, but their account-state checksums differ.
  • A related redshift/commit_snapshot_ordering failure selected an older outbox receipt after intent IDs were reused. Its receipt reader also needed the independent RedSuite slot filter, so that failure alone does not isolate the Engine bug.

Expected

Startup accounts should be inserted when absent. Reopening an existing database must preserve account data and intent IDs, so the leader and verifiers produce the same state seal after restart.

Context

The validator used Engine 0.20.0 (bc6b5a9) when the mismatch occurred. The issue is in Keeper's startup account seeding; program and sysvar startup behavior is separate.

Activity

  1. self-assigned this
    on Nov 3, 2026
  2. added theissue type on Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions