ChainSafe / ChainSafe/canton-middleware
[TRACKER] Canton ERC-20 Indexer
- Dominant language
- Go
- Stars
- 1
- Forks
- 1
- Avg merge
- 40m
- Merged PRs (30d)
- 1
Description
## Summary
Implement a **standalone ERC-20 indexer** for the Canton Network per [CIP-0086](https://github.com/canton-foundation/cips/blob/main/cip-0086/cip-0086.md). The indexer streams the Canton ledger continuously, indexes token transfer events, and serves a Canton-native HTTP query API — analogous to The Graph or SubSquid in EVM ecosystems.
The indexer replaces the current `reconciler` (polling snapshot) with a proper event-driven pipeline that captures **full transfer history** including Canton-native transfers, not just bridge mint/burn events.
---
## Motivation
The current reconciler:
- Polls `CIP56Holding` snapshots every N seconds — misses events between polls
- Has no transfer history (only current balance)
- Is coupled to the api-server process
- Not resumable — replays from offset 0 on restart
- Cannot serve a query API independently
---
## Design
Full design document: [indexer-design.md](https://github.com/user-attachments/files/25694021/indexer-design.md)
### Architecture
```
Canton Ledger API v2 (gRPC server-streaming)
│
├── pkg/cantonsdk/streaming/ NEW — generic streaming client
│ (mirrors StreamWithdrawalEvents pattern)
│
▼
pkg/indexer/fetcher/ loads checkpoint → delegates to cantonsdk/streaming
▼
pkg/indexer/parser/ decode TokenTransferEvent → MINT | BURN | TRANSFER
▼
pkg/indexer/processor/ atomic DB writes (events + balances + checkpoint)
▼
PostgreSQL (indexer.* tables)
▼
pkg/indexer/api/ HTTP :8082 — JWT auth, canton_party_id scoped
```
### Key design decisions
**1. Single unified `TokenTransferEvent` DAML template**
Add one new event template to `Events.daml` (no return type changes to existing choices):
- `MINT`: `fromParty = None`
- `BURN`: `toParty = None`
- `TRANSFER`: both set
Mirrors ERC-20 `Transfer(from, to, value)`. Emitted as a side-effect from:
- `TokenConfig.IssuerMint`
- `TokenConfig.IssuerBurn`
- `CIP56TransferFactory.transferFactory_transferImpl`
Existing `MintEvent` / `BurnEvent` kept for backward compat.
**2. Indexer is Canton-native — no `userstore` dependency**
The indexer speaks `canton_party_id`, not EVM addresses. The EVM → party_id mapping is the api-server's concern. The indexer has zero dependency on the api-server's user tables.
Auth: **JWT only** (shared JWKS with api-server). JWT must contain `canton_party_id` claim. The api-server resolves EVM address → party_id, issues a JWT, and calls the indexer.
**3. Resumable streaming via ledger offset checkpoint**
`ledger_checkpoints` table stores the last processed offset. Updated atomically inside each DB transaction. On restart, the stream resumes from this offset — no replay from scratch.
**4. Generic streaming client in cantonsdk**
`pkg/cantonsdk/streaming/` wraps `UpdateService.GetUpdates` (same gRPC call already used by `StreamWithdrawalEvents`) into a reusable, reconnect-aware subscription client.
---
## New packages / files
```
cmd/indexer/ new binary
pkg/app/indexer/server.go orchestrator (mirrors pkg/app/api/server.go)
pkg/cantonsdk/streaming/ reusable streaming client
pkg/indexer/fetcher/
pkg/indexer/parser/
pkg/indexer/processor/
pkg/indexer/store/ Bun ORM, no evm_address in token_balances
pkg/indexer/service/ query service (by party_id)
pkg/indexer/api/ HTTP handlers + JWT middleware
pkg/migrations/indexerdb/ 5 migration files
```
## DB tables (new `indexer.*` schema)
| Table | Purpose |
|---|---|
| `ledger_checkpoints` | Single-row resumption offset |
| `indexed_tokens` | Registry of indexed token contracts |
| `transfer_events` | Append-only event log (MINT/BURN/TRANSFER) |
| `token_balances` | Incremental balance cache keyed by `(party_id, token_symbol)` |
| `token_stats` | Total supply + holder count per token |
## HTTP endpoints
```
[public] GET /v1/tokens
[public] GET /v1/tokens/{symbol}/totalSupply
[JWT] GET /v1/balance/{partyID}/{symbol}
[JWT] GET /v1/transfers/{partyID}
[JWT] GET /v1/events/{partyID}
GET /health GET /metrics
```
## DAML changes required
- `Events.daml` — add `TokenTransferEvent` template
- `Config.daml` — emit from `IssuerMint`, `IssuerBurn` (no signature change)
- `TransferFactory.daml` — emit from `transferFactory_transferImpl` (no signature change)
---
## Migration from reconciler
1. Deploy indexer alongside existing api-server + reconciler
2. Validate: compare reconciler balances vs `indexer.token_balances`
3. Switch api-server token provider to call indexer API
4. Disable `StartPeriodicReconciliation`
5. Remove reconciler after one release cycle
---
## Out of scope (Phase 1)
- Canton Coin / Super Validator full-visibility mode
- GraphQL layer (`pkg/indexer/graph/`)
- WebSocket push to clients
- Multi-issuer support
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the linked indexer design and the existing StreamWithdrawalEvents pattern in pkg/cantonsdk, then review the proposed packages under pkg/indexer/ and the DAML changes in Events.daml, Config.daml, and TransferFactory.daml. Done means the standalone cmd/indexer pipeline, checkpointed PostgreSQL storage, query API, migrations, and token transfer events described here are implemented and can replace the reconciler after validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, postgresql
- Domain
- backend-api-design, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100