FilOzone / FilOzone/pdp-explorer
perf(subgraph): migrate weekly/monthly activity metrics to native timeseries + @aggregation
- Dominant language
- TypeScript
- Stars
- 3
- Forks
- 7
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 4
Description
## Problem
Almost every handler ends with a metrics fan-out via `subgraph/src/helper.ts`: `saveProviderMetrics` ×2 (weekly + monthly) + `saveProofSetMetrics` ×2 + `saveNetworkMetrics`. Each is a read-modify-write on a **mutable** entity, so a single on-chain event produces ~5 extra store reads and ~5 new mutable row versions — the single largest write amplifier in the subgraph. The `NetworkMetric` singleton in particular gets a new row version for nearly every event. The generic `store.get`/`Entity`/`Value` plumbing with `toHexString()` string keys adds overhead on top.
## Proposal
We're on `specVersion 1.3.0`, so graph-node native aggregations are available:
- Define timeseries entities (`@entity(timeseries: true)`, `id: Int8!`, `timestamp: Timestamp!`) for provider activity and dataset activity data points. Handlers **append one immutable point** instead of 4 read-modify-writes; graph-node rolls up buckets in Postgres at interval boundaries.
- Define `@aggregation(intervals: ["hour", "day"], source: ...)` entities with`@aggregate(fn: "sum", arg: ...)` for the existing counters (rootsAdded/removed, dataSize, proofs, faults, …).
- Replace `NetworkMetric` running totals with `cumulative: true` aggregates from the same source, removing the singleton churn.
- Delete `helper.ts` (`saveProviderMetrics` / `saveProofSetMetrics` / `saveNetworkMetrics`) once migrated.
## Constraints / client impact
- Aggregation intervals are **hour/day only** — no native week/month. The client (`subgraph-client/src/utility/queries.ts`: `weeklyProviderActivities`, `weeklyProofSetActivities`, `monthlyProofSetActivities`, `networkMetricsQuery`) must sum daily buckets into weeks/months, or we keep a thin weekly/monthly layer and only migrate the network totals.
- Full resync + coordinated client release required.
## Acceptance criteria
- [ ] Schema design reviewed (entity + aggregation definitions) before implementation
- [ ] Handlers perform at most 1 metrics write per event (immutable append)
- [ ] Client charts/stat tiles render identically against the new schema
## Reference
https://thegraph.com/docs/en/subgraphs/best-practices/timeseries
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing subgraph/src/helper.ts and the existing entity schema, then inspect subgraph-client/src/utility/queries.ts and the weeklyProviderActivities, weeklyProofSetActivities, monthlyProofSetActivities, and networkMetricsQuery entry points. Confirm the schema design before implementation. Done means handlers use at most one immutable metrics write per event and client charts and stat tiles render identically after a full resync.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, typescript
- Domain
- backend, data, databases
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100