Skip to content

perf(subgraph): migrate weekly/monthly activity metrics to native timeseries + @aggregation #139

Description

@silent-cipher

Problem

Almost every handler ends with a metrics fan-out via subgraph/src/helper.ts: saveProviderMetrics ×2 (weekly + monthly) + saveProofSetMetrics ×2 + saveNetworkMetrics. Each is a read-modify-write on a mutable entity, so a single on-chain event produces ~5 extra store reads and ~5 new mutable row versions — the single largest write amplifier in the subgraph. The NetworkMetric singleton in particular gets a new row version for nearly every event. The generic store.get/Entity/Value plumbing with toHexString() string keys adds overhead on top.

Proposal

We're on specVersion 1.3.0, so graph-node native aggregations are available:

  • Define timeseries entities (@entity(timeseries: true), id: Int8!, timestamp: Timestamp!) for provider activity and dataset activity data points. Handlers append one immutable point instead of 4 read-modify-writes; graph-node rolls up buckets in Postgres at interval boundaries.
  • Define @aggregation(intervals: ["hour", "day"], source: ...) entities with@aggregate(fn: "sum", arg: ...) for the existing counters (rootsAdded/removed, dataSize, proofs, faults, …).
  • Replace NetworkMetric running totals with cumulative: true aggregates from the same source, removing the singleton churn.
  • Delete helper.ts (saveProviderMetrics / saveProofSetMetrics / saveNetworkMetrics) once migrated.

Constraints / client impact

  • Aggregation intervals are hour/day only — no native week/month. The client (subgraph-client/src/utility/queries.ts: weeklyProviderActivities, weeklyProofSetActivities, monthlyProofSetActivities, networkMetricsQuery) must sum daily buckets into weeks/months, or we keep a thin weekly/monthly layer and only migrate the network totals.
  • Full resync + coordinated client release required.

Acceptance criteria

  • Schema design reviewed (entity + aggregation definitions) before implementation
  • Handlers perform at most 1 metrics write per event (immutable append)
  • Client charts/stat tiles render identically against the new schema

Reference

https://thegraph.com/docs/en/subgraphs/best-practices/timeseries

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    • Status
      🐱 Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions