FilOzone / FilOzone/pdp-explorer

perf(subgraph): migrate weekly/monthly activity metrics to native timeseries + @aggregation

Open
#139 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
3
Forks
7
Avg merge
1d 4h
Merged PRs (30d)
4

Description

## Problem

Almost every handler ends with a metrics fan-out via `subgraph/src/helper.ts`: `saveProviderMetrics` ×2 (weekly + monthly) + `saveProofSetMetrics` ×2 + `saveNetworkMetrics`. Each is a read-modify-write on a **mutable** entity, so a single on-chain event produces ~5 extra store reads and ~5 new mutable row versions — the single largest write amplifier in the subgraph. The `NetworkMetric` singleton in particular gets a new row version for nearly every event. The generic `store.get`/`Entity`/`Value` plumbing with `toHexString()` string keys adds overhead on top.

## Proposal

We're on `specVersion 1.3.0`, so graph-node native aggregations are available:

- Define timeseries entities (`@entity(timeseries: true)`, `id: Int8!`, `timestamp: Timestamp!`) for provider activity and dataset activity data points. Handlers **append one immutable point** instead of 4 read-modify-writes; graph-node rolls up buckets in Postgres at interval boundaries.
- Define `@aggregation(intervals: ["hour", "day"], source: ...)` entities with`@aggregate(fn: "sum", arg: ...)` for the existing counters (rootsAdded/removed, dataSize, proofs, faults, …).
- Replace `NetworkMetric` running totals with `cumulative: true` aggregates from the same source, removing the singleton churn.
- Delete `helper.ts` (`saveProviderMetrics` / `saveProofSetMetrics` / `saveNetworkMetrics`) once migrated.

## Constraints / client impact

- Aggregation intervals are **hour/day only** — no native week/month. The client (`subgraph-client/src/utility/queries.ts`: `weeklyProviderActivities`, `weeklyProofSetActivities`, `monthlyProofSetActivities`, `networkMetricsQuery`) must sum daily buckets into weeks/months, or we keep a thin weekly/monthly layer and only migrate the network totals.
- Full resync + coordinated client release required.

## Acceptance criteria

- [ ] Schema design reviewed (entity + aggregation definitions) before implementation
- [ ] Handlers perform at most 1 metrics write per event (immutable append)
- [ ] Client charts/stat tiles render identically against the new schema

## Reference

https://thegraph.com/docs/en/subgraphs/best-practices/timeseries

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing subgraph/src/helper.ts and the existing entity schema, then inspect subgraph-client/src/utility/queries.ts and the weeklyProviderActivities, weeklyProofSetActivities, monthlyProofSetActivities, and networkMetricsQuery entry points. Confirm the schema design before implementation. Done means handlers use at most one immutable metrics write per event and client charts and stat tiles render identically after a full resync.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, typescript
Domain
backend, data, databases
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.