Skip to content

Full-history query benchmark commands — reads across all data types  #856

Description

@marwen-abid

TL;DR

#771 was re-scoped to ingestion benchmarks; this issue carries the query half. Rewire the rpc-hack read benches onto the production store readers as a bench-query subcommand next to bench-ingest, reusing #771's CSV sink and reporting format. Together with #771 this completes the original scope (and #730's).

Scope

  • The eight read benches from the prototype: {hot,cold} × {ledgers, txpage, txhash, events}.
  • Measure the production read paths in internal/fullhistory/storage/stores/ — e.g. eventstore/query.go + cold_reader.go, ledger/cold_reader.go + hot_store.go, txhash/read_assembly.go + cold_reader.go — not the prototype's own reader implementations (same rewiring principle as Full-history benchmark commands — ingest across all data types #771).
  • Cold event/txhash benches must measure the complete correct read, including the MPHF post-filter — an index probe alone benchmarks a correctness bug, not a query.
  • --query-concurrency sweeps and the grid runner (concurrency × data type matrix).
  • Events query-corpus auto-generation: derive realistic contract-ID/topic terms from the ingested data rather than hand-picked constants.
  • XDR-views only (the decided path; no views-vs-decode comparison).
  • CSV output in Full-history benchmark commands — ingest across all data types #771's schema (stage,n,n_items,total_ns,p50_ns,p90_ns,p99_ns,max_ns) so Run full-history benchmark commands in CI to catch perf regressions #775 can consume ingest and query reports uniformly.
  • Land on feature/full-history at mergeable quality.

Source

rpc-hack scripts/bench-fullhistory/: bench_{cold,hot}_*.go, bench_grid.go, bench_concurrent_runner.go, corpus.go, and the cache-eviction helpers (cold read numbers are meaningless against a warm page cache). Carry the harness and methodology; leave the prototype's reader code behind.

Reuses from #771

internal/fullhistory/bench/ — the cobra subcommand pattern, the CSV/percentile sink, and the fixture/source plumbing. Benchmark inputs (a populated hot DB / frozen cold chunks) can be produced by bench-ingest itself.

Out of scope

Dependencies

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions