Skip to content

[DRAFT] Optimize RPCv1 getEvents query-path - #1002

Draft
cjonas9 wants to merge 56 commits into
feature/full-historyfrom
optimize-getEvents-scratchpad
Draft

cjonas9 wants to merge 56 commits into
feature/full-historyfrom
optimize-getEvents-scratchpad

Conversation

@cjonas9

@cjonas9 cjonas9 commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

What

This branch is not currently intended for any review beyond cursory curiosity-satisfying glances. Despite the optimizations made by XDR views, getEvents remained among the slowest of our endpoints. Several low-hanging fruit optimizations came up organically while I worked on this, and this branch is intended to house them temporarily:

  • stellar-rpc/cmd/stellar-rpc/internal/rpcv1/sqlitedb/event.go::GetEvents(): the SQL statement pulling event rows is now limited to page size. It recurses whenever the batch is exhausted but the scanner still wants rows. (needed because the Go post-filter can reject rows SQL accepted).
  • New composite index on (contractId, topic1, id) in rpcv1/sqlitedb/sqlmigrations/07_contract_topic1_index.sql: this is cheeky, but very few lines of code, and relatively little extra data stored. Contract-plus-topic is an unbelievably common filter shape, and to serve queries of that type, our backend uses whichever single-column index the planner guesses, then filters the other column row by row. That is very slow and is solved by this.
    • I argue this is not overfitting. topic1 is the verb, covering fee, transfer, mint, burn. In a typical 7 day DB those four values cover >99% of all events, and there are only ~400 distinct topic1 values total.

Why

Despite optimizations, GetEvents improved extremely unremarkably. Here are the results of its performance in recent CI release eval runs:

Run 1:

Endpoint Target RPS Requests Errors p50 (ms) p95 (ms) p99 (ms) p99.9 (ms)
getEvents 75 11222 6 (0.1%) 0.8 13.0 73.7 3670.0
Run 1 results extended
Endpoint Target RPS Requests Errors p50 (ms) p95 (ms) p99 (ms) p99.9 (ms)
getEvents/catch-up 3 429 0 (0.0%) 5.1 28.9 83.1 116.7
getEvents/deep-pager 17.25 2560 1 (0.0%) 0.7 40.6 137.6 1754.1
getEvents/deep-scan 2.25 373 1 (0.3%) 0.7 7.0 12.0 10002.4
getEvents/firehose 1.5 222 0 (0.0%) 0.8 4.3 31.2 131.1
getEvents/head-poll 36 5444 0 (0.0%) 0.8 6.8 24.5 96.6
getEvents/tail-poll 9 1310 4 (0.3%) 0.7 1.3 1019.9 10002.4
getEvents/transfer-watcher 6 884 0 (0.0%) 9.9 17.1 39.1 121.0

Run 2:

Endpoint Target RPS Requests Errors p50 (ms) p95 (ms) p99 (ms) p99.9 (ms)
getEvents 75 11223 5 (0.0%) 1.2 14.2 85.5 2310.1
Run 2 results extended
Endpoint Target RPS Requests Errors p50 (ms) p95 (ms) p99 (ms) p99.9 (ms)
getEvents/catch-up 3 456 0 (0.0%) 6.2 27.5 57.0 123.3
getEvents/deep-pager 17.25 2586 0 (0.0%) 1.1 40.0 151.3 1305.6
getEvents/deep-scan 2.25 315 1 (0.3%) 1.1 9.8 47.9 10002.4
getEvents/firehose 1.5 220 0 (0.0%) 1.1 5.3 9.1 137.2
getEvents/head-poll 36 5378 0 (0.0%) 1.2 7.7 38.2 128.6
getEvents/tail-poll 9 1378 4 (0.3%) 1.1 1.7 985.1 10002.4
getEvents/transfer-watcher 6 890 0 (0.0%) 10.4 18.1 24.1 90.0

Pre-XDR Views results:

Run 1:

Endpoint Target RPS Requests Errors p50 (ms) p95 (ms) p99 (ms) p99.9 (ms)
getEvents 75 11224 10 (0.1%) 1.0 12.5 72.7 9289.7
above results extended
Endpoint Target RPS Requests Errors p50 (ms) p95 (ms) p99 (ms) p99.9 (ms)
getEvents/catch-up 3 484 0 (0.0%) 9.9 40.4 63.2 108.5
getEvents/deep-pager 17.25 2582 1 (0.0%) 1.0 44.3 125.0 3086.3
getEvents/deep-scan 2.25 336 3 (0.9%) 1.0 11.0 61.0 10010.6
getEvents/firehose 1.5 219 0 (0.0%) 1.0 6.7 30.4 113.2
getEvents/head-poll 36 5421 0 (0.0%) 1.0 6.9 23.4 101.7
getEvents/tail-poll 9 1344 6 (0.4%) 1.0 1.7 1221.6 10002.4
getEvents/transfer-watcher 6 838 0 (0.0%) 7.7 14.1 18.4 31.2

Run 2:

Endpoint Target RPS Requests Errors p50 (ms) p95 (ms) p99 (ms) p99.9 (ms)
getEvents 75 8999 4 (0.0%) 0.8 14.6 82.8 1669.1
above results extended
Endpoint Target RPS Requests Errors p50 (ms) p95 (ms) p99 (ms) p99.9 (ms)
getEvents/catch-up 3 331 0 (0.0%) 10.2 36.2 63.1 99.7
getEvents/deep-pager 17.25 2018 0 (0.0%) 0.8 45.2 129.5 1264.6
getEvents/deep-scan 2.25 274 1 (0.4%) 0.7 12.1 58.4 10002.4
getEvents/firehose 1.5 180 0 (0.0%) 0.8 6.7 30.9 133.1
getEvents/head-poll 36 4374 0 (0.0%) 0.8 7.3 63.6 115.1
getEvents/tail-poll 9 1100 3 (0.3%) 0.7 1.1 897.0 10002.4
getEvents/transfer-watcher 6 722 0 (0.0%) 10.1 17.2 73.7 152.4

Known limitations

N/A

@cjonas9
cjonas9 changed the base branch from main to feature/full-history September 10, 2026 20:30
@cjonas9
cjonas9 changed the base branch from feature/full-history to getEvents-differential-test September 10, 2026 20:30
@github-actions

Copy link
Copy Markdown
Contributor

🧪 Performance Evaluation Test #1

Commit: f0acd784fcf6 (optimize-getEvents-scratchpad)
Run: https://github.com/stellar/stellar-rpc/actions/runs/34527078976

❌ Apply-load ingestion — verdict: none

No result object published (leg timed out or failed before publishing). See the run logs.

✅ Backfill ingestion — verdict: ok

⏳ Backfill ingestion — f0acd784fcf6

Metric Value
Ledgers ingested 120960 ([64248704 -> 64369663])
Retention window 120960
Wall-clock (total) 3h42m12s
Ingest phase 2h36m14s
Bulk-load finalize phase 1h5m58s
Ledgers/sec (ingest) 12.9

✅ Endpoint load test — verdict: ok

🎯 Endpoint load test — f0acd784fcf6

Serial blast per endpoint (ramp-up 1m, duration 3m, error kill switch 75%, blaster aadc1a17595f) against the backfilled RPC (ledgers [64249778, 64370737], handoff wait 1561s).

Endpoint Target RPS Requests Errors p50 (ms) p95 (ms) p99 (ms) p99.9 (ms)
getEvents 75 11224 0 (0.0%) 0.9 13.5 91.6 146.8
getFeeStats 250 37492 0 (0.0%) 0.6 0.7 70.0 137.2
getHealth 250 37493 0 (0.0%) 0.5 0.7 59.7 127.0
getLatestLedger 15 2245 0 (0.0%) 94.7 131.0 185.6 242.8
getLedgers (limit=5) 3 442 0 (0.0%) 607.2 1180.7 1594.4 1868.8
getNetwork 100 14973 0 (0.0%) 25.2 348.9 583.2 652.8
getTransaction 75 11225 0 (0.0%) 6.6 19.0 62.4 139.9
getTransactions (limit=200) 10 1473 0 (0.0%) 92.7 294.9 414.5 470.8
getVersionInfo 100 14851 0 (0.0%) 24.4 136.8 183.9 220.8
getEvents results extended
Endpoint Target RPS Requests Errors p50 (ms) p95 (ms) p99 (ms) p99.9 (ms)
getEvents/catch-up 3 457 0 (0.0%) 8.1 30.4 80.5 139.6
getEvents/deep-pager 17.25 2555 0 (0.0%) 0.9 42.7 118.4 195.5
getEvents/deep-scan 2.25 338 0 (0.0%) 0.9 8.3 82.4 105.7
getEvents/firehose 1.5 208 0 (0.0%) 0.9 4.7 82.4 103.9
getEvents/head-poll 36 5367 0 (0.0%) 0.9 4.4 73.5 141.8
getEvents/tail-poll 9 1391 0 (0.0%) 0.9 1.2 77.8 140.5
getEvents/transfer-watcher 6 908 0 (0.0%) 5.9 11.3 57.6 115.5

✅ Go endpoint benchmarks — verdict: ok

Baseline v28.0.1 (273f19e4fcb1) vs candidate f0acd784fcf6-benchmem -count=10, both refs sequentially on one box; rpcv2 excluded.

benchstat: baseline vs candidate
goos: linux
goarch: amd64
pkg: github.com/stellar/stellar-rpc/cmd/stellar-rpc/internal/db
cpu: Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
                  │ baseline.txt │
                  │    sec/op    │
GetLedgerRange-8     956.7n ± 1%
BatchGetLedgers-8    747.3µ ± 2%
geomean              26.74µ

                  │ baseline.txt │
                  │     B/op     │
GetLedgerRange-8      16.00 ± 0%
BatchGetLedgers-8   520.8Ki ± 0%
geomean             2.853Ki

                  │ baseline.txt │
                  │  allocs/op   │
GetLedgerRange-8      4.000 ± 0%
BatchGetLedgers-8    2.314k ± 0%
geomean               96.21

pkg: github.com/stellar/stellar-rpc/cmd/stellar-rpc/internal/feewindow
                                                           │ baseline.txt │
                                                           │    sec/op    │
ComputeFeeDistribution/computeFeeDistribution-8               9.625µ ± 0%
ComputeFeeDistribution/alternativeComputeFeeDistribution-8    219.4µ ± 3%
geomean                                                       45.96µ

                                                           │  baseline.txt  │
                                                           │      B/op      │
ComputeFeeDistribution/computeFeeDistribution-8                0.000 ± 0%
ComputeFeeDistribution/alternativeComputeFeeDistribution-8   565.4Ki ± 0%
geomean                                                                   ¹
¹ summaries must be >0 to compute geomean

                                                           │ baseline.txt │
                                                           │  allocs/op   │
ComputeFeeDistribution/computeFeeDistribution-8              0.000 ± 0%
ComputeFeeDistribution/alternativeComputeFeeDistribution-8   30.00 ± 0%
geomean                                                                 ¹
¹ summaries must be >0 to compute geomean

pkg: github.com/stellar/stellar-rpc/cmd/stellar-rpc/internal/methods
                               │  baseline.txt  │               candidate.txt               │
                               │     sec/op     │     sec/op      vs base                   │
GetEventsTopicFilters-8           2.592m ± 1%      2.335m ± 1%      -9.92% (p=0.000 n=10)
GetEvents-8                       163.6µ ±  ∞ ¹    131.9µ ±  ∞ ¹         ~ (p=1.000 n=1)  ²
GetLedgers-8                     1349.6µ ±  ∞ ¹    842.3µ ±  ∞ ¹         ~ (p=1.000 n=1)  ²
JSONTransactions/JSON_format-8    7.937µ ± 1%     14.087µ ± 0%     +77.48% (p=0.000 n=10)
JSONTransactions/XDR_format-8     2.585µ ± 0%      7.593µ ± 0%    +193.79% (p=0.000 n=10)
GetProtocolVersion-8              43.52µ ±  ∞ ¹    37.27µ ±  ∞ ¹         ~ (p=1.000 n=1)  ²
geomean                           89.41µ           100.6µ          +12.47%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                               │  baseline.txt   │               candidate.txt               │
                               │      B/op       │      B/op       vs base                   │
GetEventsTopicFilters-8          190.73Ki ± 0%     99.94Ki ± 0%     -47.60% (p=0.000 n=10)
GetEvents-8                       43.16Ki ±  ∞ ¹   32.73Ki ±  ∞ ¹         ~ (p=1.000 n=1)  ²
GetLedgers-8                     1402.2Ki ±  ∞ ¹   910.4Ki ±  ∞ ¹         ~ (p=1.000 n=1)  ²
JSONTransactions/JSON_format-8    2.000Ki ± 0%     5.438Ki ± 0%    +171.88% (p=0.000 n=10)
JSONTransactions/XDR_format-8     1.336Ki ± 0%     4.781Ki ± 0%    +257.89% (p=0.000 n=10)
GetProtocolVersion-8              7.909Ki ±  ∞ ¹   7.689Ki ±  ∞ ¹         ~ (p=1.000 n=1)  ²
geomean                           25.00Ki          29.00Ki          +16.04%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                               │ baseline.txt  │              candidate.txt               │
                               │   allocs/op   │   allocs/op    vs base                   │
GetEventsTopicFilters-8          1523.5 ± 0%      781.0 ± 0%     -48.74% (p=0.000 n=10)
GetEvents-8                       621.0 ±  ∞ ¹    481.0 ±  ∞ ¹         ~ (p=1.000 n=1)  ²
GetLedgers-8                     4.723k ±  ∞ ¹   2.710k ±  ∞ ¹         ~ (p=1.000 n=1)  ²
JSONTransactions/JSON_format-8    25.00 ± 0%      49.00 ± 0%     +96.00% (p=0.000 n=10)
JSONTransactions/XDR_format-8     23.00 ± 0%      47.00 ± 0%    +104.35% (p=0.000 n=10)
GetProtocolVersion-8              122.0 ±  ∞ ¹    119.0 ±  ∞ ¹         ~ (p=1.000 n=1)  ²
geomean                           260.6           255.6           -1.92%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

pkg: github.com/stellar/stellar-rpc/cmd/stellar-rpc/internal/preflight
               │ baseline.txt │           candidate.txt            │
               │    sec/op    │   sec/op     vs base               │
GetPreflight-8    814.4µ ± 1%   793.0µ ± 1%  -2.63% (p=0.000 n=10)

               │ baseline.txt │         candidate.txt          │
               │     B/op     │     B/op      vs base          │
GetPreflight-8   52.10Ki ± 0%   52.10Ki ± 0%  ~ (p=0.596 n=10)

               │ baseline.txt │        candidate.txt         │
               │  allocs/op   │ allocs/op   vs base          │
GetPreflight-8     290.0 ± 0%   290.0 ± 0%  ~ (p=1.000 n=10)

pkg: github.com/stellar/stellar-rpc/cmd/stellar-rpc/internal/rpcv1/sqlitedb
                                 │ candidate.txt │
                                 │    sec/op     │
GetLedgerRange-8                     956.4n ± 2%
BatchGetLedgers-8                    280.3µ ± 2%
OldestLedgerRangeLookup/512KiB-8     174.2µ ± 1%
OldestLedgerRangeLookup/2MiB-8       591.9µ ± 1%
OldestLedgerRangeLookup/4MiB-8       1.157m ± 1%
geomean                              126.2µ

                                 │ candidate.txt │
                                 │     B/op      │
GetLedgerRange-8                      16.00 ± 0%
BatchGetLedgers-8                   288.5Ki ± 0%
OldestLedgerRangeLookup/512KiB-8    7.503Ki ± 0%
OldestLedgerRangeLookup/2MiB-8      7.503Ki ± 0%
OldestLedgerRangeLookup/4MiB-8      7.503Ki ± 0%
geomean                             4.528Ki

                                 │ candidate.txt │
                                 │   allocs/op   │
GetLedgerRange-8                      4.000 ± 0%
BatchGetLedgers-8                    1.319k ± 0%
OldestLedgerRangeLookup/512KiB-8      93.00 ± 0%
OldestLedgerRangeLookup/2MiB-8        93.00 ± 0%
OldestLedgerRangeLookup/4MiB-8        93.00 ± 0%
geomean                               84.25

pkg: github.com/stellar/stellar-rpc/cmd/stellar-rpc/internal/store
                                                           │ candidate.txt │
                                                           │    sec/op     │
ComputeFeeDistribution/alternativeComputeFeeDistribution-8     204.5µ ± 3%
ComputeFeeDistribution/ComputeFeeDistribution-8                9.135µ ± 0%
geomean                                                        43.22µ

                                                           │ candidate.txt  │
                                                           │      B/op      │
ComputeFeeDistribution/alternativeComputeFeeDistribution-8   565.4Ki ± 0%
ComputeFeeDistribution/ComputeFeeDistribution-8                0.000 ± 0%
geomean                                                                   ¹
¹ summaries must be >0 to compute geomean

                                                           │ candidate.txt │
                                                           │   allocs/op   │
ComputeFeeDistribution/alternativeComputeFeeDistribution-8    30.00 ± 0%
ComputeFeeDistribution/ComputeFeeDistribution-8               0.000 ± 0%
geomean                                                                  ¹
¹ summaries must be >0 to compute geomean

Raw benchmark logs (s3://stellar-rpc-ci-load-test/runs/34527078976/go-bench/): baseline.txt, benchstat.txt, candidate.txt

cjonas9 and others added 27 commits September 11, 2026 03:27
getTransactions costs ~100ms per limit=50 page, and a CPU profile over 40
such requests attributes 1.83s cum to xdr.LedgerEntryChanges.DecodeFrom plus
~2.9s of GC drain/scan on top: the shared handler fully XDR-unmarshals the
entire multi-MB LedgerCloseMeta once per ledger it walks, to read a handful of
per-transaction fields off it. It has no choice — the serving interface only
offers GetLedger, which returns a decoded xdr.LedgerCloseMeta — yet BOTH
backends hold the ledger as raw XDR bytes and decode on the way out.

Add the seam that lets a handler skip that decode:

	WithLedgerRaw(ctx, sequence, fn) (found bool, err error)

on store.LedgerReaderTx — the Tx/walk variant, because the paging path is the
only caller and the one-shot store.LedgerReader has none. Loan-shaped rather
than returning bytes: the v2 backend can then hand over its chunk reader's
scratch buffer directly, with no copy and no lifetime question, since the
bytes' validity ends with fn. The contract is documented on the interface —
valid only inside fn, read-only, must not be retained — alongside the
walk-cursor rule the two accessors now share.

The backends implement it as pass-throughs of what they already hold:

- rpcv2 adapter: GetLedger's walk (deadline check, window gate, priming, the
  two loud contract failures) moves into a shared walkTo, so the decoding and
  raw accessors advance one cursor and cannot drift on where the walk is.
  WithLedgerRaw is walkTo plus fn — no decode at all.

- rpcv1 sqlitedb: selects the same `meta` column of ledger_close_meta into a
  []byte instead of an xdr.LedgerCloseMeta, so the row scan never runs
  UnmarshalBinary. The lent bytes are ours to lend: scanning a BLOB into a
  *[]byte goes through database/sql's convertAssign, which clones the driver's
  row buffer (only sql.RawBytes opts out), and BatchGetLedgers already depends
  on exactly that by retaining the slices it selects.

GetLedger stays: every other handler still uses it. No caller uses the new
seam yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7opQUW9UZPKzE6B3tz9b5
(cherry picked from commit d8e0797)
adapters.transactionFromView turns an ingest.LedgerTransactionView into the
store.Transaction the serving handlers format. The shared getTransactions
handler is about to need exactly that reshape, and a second copy of it in
internal/methods would be a copy that can silently disagree with this one.

Move it to internal/store as TransactionFromView, next to ParseTransaction —
its parsed-path parallel — and have the adapter call it. Behavior is
unchanged: same fields, same aliasing, same caller-side rule about only
handing it a view whose bytes were already copied out (that note moves to the
adapter's call site, which is where the guarantee actually holds).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7opQUW9UZPKzE6B3tz9b5
(cherry picked from commit ff21f43)
…ded LCM

A limit=50 getTransactions page costs ~100ms. Profiling 40 of them attributes
1.83s cum to xdr.LedgerEntryChanges.DecodeFrom and ~2.9s to GC drain/scan on
top of it, and neither is work the response needs: the page loop fully
XDR-unmarshaled every ledger it touched — LedgerEntryChanges, bucket entries,
the lot — to read a handful of per-transaction fields back out and base64 them
straight to the client.

Read the ledger through the SDK's zero-copy views instead. Each ledger is now
borrowed as raw bytes via the new store.LedgerReaderTx.WithLedgerRaw, and
ingest.LedgerTransactionViewRange walks its TxProcessing to materialize only
the page's worth of transactions — nothing is unmarshaled, and the walk stops
at the page limit rather than at the ledger's end. store.TransactionFromView
reshapes each one into the store.Transaction the response renderer already
takes, so the whole formatting half of the handler is untouched (it moves into
transactionInfo, shared verbatim with the differential's reference below). The
same precedent already serves getTransaction by hash in the v2 tree
(rpcv2/stores/txhash).

One field does not survive the move unaided. For a TransactionMeta V3 whose
envelope declares Soroban data, the parsed reader always reports exactly one
operation slice, even when the meta carries no SorobanMeta at all
(GetTransactionEvents: OperationEvents = make([][]xdr.ContractEvent, 1)),
while the view extractor leaves it empty — so contractEventsXdr would come
back as [] where it used to be [[]]. stellar-core emits exactly that shape for
a Soroban transaction charged but never executed, so it is reachable on real
protocol 20-22 history. repairV3OperationArity restores it with one
union-discriminant read of the meta plus, only for a V3 meta that came back
with no operations, one view walk of the envelope for the Soroban flag the SDK
computes internally and does not expose. Still no decode. Everything else —
envelope, result, meta, diagnostic/transaction/contract events, hash, apply
order, fee-bump flag, ledger sequence and close time — comes off the views
byte for byte.

The correctness argument is the differential: the pre-change extraction is
reconstructed in get_transactions_differential_test.go and both paths are run
over one sqlite corpus that sweeps LCM V1/V2, TransactionMeta V1/V3/V4,
classic and Soroban and fee-bump envelopes, present/absent/empty SorobanMeta,
per-operation and top-level and diagnostic events, empty ledgers, five-tx
ledgers, both response formats, page limits that land mid-ledger and on
boundaries, explicit cursors, and a full cursor round-trip where each path
drives its own cursor chain. Every comparison is json.Marshal byte equality of
the whole GetTransactionsResponse (not JSONEq — a reordered or vanished field
must fail). ~410 subtests; a corpus-vacuity guard and a renderer field-mapping
pin keep them from passing for the wrong reason.

Every pre-existing getTransactions test passes unmodified; the only edit to
one was giving the sparseLedgerReader stub the new interface method.

Behavior note: an unreadable ledger's extraction error is now InternalError
throughout. Before, a failure inside the transaction reader (only reachable on
a corrupt stored ledger) surfaced as InvalidParams while the same corruption
caught by the reader's constructor surfaced as InternalError; the view path
cannot tell those apart, and a corrupt stored ledger is not the client's
fault.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7opQUW9UZPKzE6B3tz9b5
(cherry picked from commit 2042a38)

[cherry-pick resolution: repairV3OperationArity moved from methods into
store.ParseTransaction so the v1 and v2 by-hash view reads are repaired too,
not only the pager; TransactionFromView folded into the existing view-shaped
store.ParseTransaction; the differential's legacy reference extraction is
frozen test-locally as legacyParseTransaction since the decode-based
store.ParseTransaction no longer exists on this branch.]
The pager now borrows raw ledgers through WithLedgerRaw, so LedgerReaderTx's
owned-copy GetLedgerView has no callers left: remove it from the interface,
both backends, and the mocks/stubs.

Revert the V3 fixture edits (and the getLedgers goldens derived from them):
a Soroban envelope with absent SorobanMeta is real protocol 20-22 history,
and the old fixtures pin the [[]] contractEventsXdr arity that
repairV3OperationArity (in store.ParseTransaction) now preserves on the view
path. Both fixtures die with that repair at the SDK pin bump for
stellar/go-stellar-sdk#5997.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
getevents-v2 took main (go-stellar-sdk#5997 included), so the pin moves to
2cc23142 and the SDK now aligns the V3-no-SorobanMeta operation arity itself:
delete repairV3OperationArity, envelopeIsSoroban, and their unit test, and
return store.ParseTransaction to its plain single-value reshape. The fixture
and differential pins for the [[]] shape stay and now pass on the SDK alone.

The bump also picks up the protocols/rpc EventInfo.InSuccessfulContractCall
deletion (go-stellar-sdk#5995): drop the field from getEvents' response
assembly and test expectations. xdr.DiagnosticEvent's field is unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
getTransactions costs ~100ms per limit=50 page, and a CPU profile over 40
such requests attributes 1.83s cum to xdr.LedgerEntryChanges.DecodeFrom plus
~2.9s of GC drain/scan on top: the shared handler fully XDR-unmarshals the
entire multi-MB LedgerCloseMeta once per ledger it walks, to read a handful of
per-transaction fields off it. It has no choice — the serving interface only
offers GetLedger, which returns a decoded xdr.LedgerCloseMeta — yet BOTH
backends hold the ledger as raw XDR bytes and decode on the way out.

Add the seam that lets a handler skip that decode:

	WithLedgerRaw(ctx, sequence, fn) (found bool, err error)

on store.LedgerReaderTx — the Tx/walk variant, because the paging path is the
only caller and the one-shot store.LedgerReader has none. Loan-shaped rather
than returning bytes: the v2 backend can then hand over its chunk reader's
scratch buffer directly, with no copy and no lifetime question, since the
bytes' validity ends with fn. The contract is documented on the interface —
valid only inside fn, read-only, must not be retained — alongside the
walk-cursor rule the two accessors now share.

The backends implement it as pass-throughs of what they already hold:

- rpcv2 adapter: GetLedger's walk (deadline check, window gate, priming, the
  two loud contract failures) moves into a shared walkTo, so the decoding and
  raw accessors advance one cursor and cannot drift on where the walk is.
  WithLedgerRaw is walkTo plus fn — no decode at all.

- rpcv1 sqlitedb: selects the same `meta` column of ledger_close_meta into a
  []byte instead of an xdr.LedgerCloseMeta, so the row scan never runs
  UnmarshalBinary. The lent bytes are ours to lend: scanning a BLOB into a
  *[]byte goes through database/sql's convertAssign, which clones the driver's
  row buffer (only sql.RawBytes opts out), and BatchGetLedgers already depends
  on exactly that by retaining the slices it selects.

GetLedger stays: every other handler still uses it. No caller uses the new
seam yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7opQUW9UZPKzE6B3tz9b5
(cherry picked from commit d8e0797)
The pager now borrows raw ledgers through WithLedgerRaw, so LedgerReaderTx's
owned-copy GetLedgerView has no callers left: remove it from the interface,
both backends, and the mocks/stubs.

Revert the V3 fixture edits (and the getLedgers goldens derived from them):
a Soroban envelope with absent SorobanMeta is real protocol 20-22 history,
and the old fixtures pin the [[]] contractEventsXdr arity that
repairV3OperationArity (in store.ParseTransaction) now preserves on the view
path. Both fixtures die with that repair at the SDK pin bump for
stellar/go-stellar-sdk#5997.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@cjonas9
cjonas9 force-pushed the getEvents-differential-test branch from c8067bc to 9ed0bc9 Compare September 11, 2026 17:40
@cjonas9
cjonas9 force-pushed the optimize-getEvents-scratchpad branch from f0acd78 to f79f0cc Compare September 11, 2026 17:46
@cjonas9
cjonas9 force-pushed the getEvents-differential-test branch 11 times, most recently from 75fb031 to a8121f6 Compare September 17, 2026 22:10
@cjonas9
cjonas9 force-pushed the getEvents-differential-test branch from a8121f6 to 5e66095 Compare September 18, 2026 21:23
Base automatically changed from getEvents-differential-test to feature/full-history September 21, 2026 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants