ChainSafe / ChainSafe/canton-middleware

Postgres-backed nonce store for multi-replica api-server

Open
#365 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
1
Forks
1
Avg merge
40m
Merged PRs (30d)
1

Description

## Context

The SIWE login flow introduced in #353 stores login nonces in an in-process map (`pkg/auth/service/nonce_provider/in_memory.go`, `InMemory`). This is explicitly documented as single-replica only:

> `InMemory` is an in-process nonce store suitable for a single api-server replica. [...] run more than one replica and this must be replaced with a shared (e.g. Postgres) store.

The `NonceStore` interface is already the seam for this:

```go
type NonceStore interface {
Issue(address string) (string, error)
Consume(nonce string) bool
}
```

## Problem

With more than one api-server instance behind a load balancer, `GET /auth/nonce` and the SIWE verify step can land on different replicas. A nonce issued by replica A is invisible to replica B, so `Consume` returns false and login fails intermittently. This blocks horizontal scaling / HA for the api-server.

## Proposal

Add a Postgres-backed `NonceStore` implementation so nonce state is shared across replicas.

- New implementation (e.g. `pkg/auth/service/nonce_provider/postgres.go`) satisfying the existing `NonceStore` interface — no changes needed to `login.go` / handlers.
- Bun-backed DAO + migration under `pkg/migrations/apidb/` following the existing `mghelper.CreateSchema` / `DropTables` pattern.
- Table keyed by address (one live nonce per address, matching current semantics), with `nonce`, `expiry`, and a reverse lookup on `nonce` for `Consume`.
- `Consume` must be atomic (delete-and-return-whether-live in a single statement / transaction) so a nonce can only be consumed once even under concurrent verify requests across replicas.
- Preserve current behavior: reuse a live nonce per address on repeat `Issue`; capacity guard (reject rather than evict a live nonce); TTL-based expiry with periodic purge (or lazy purge / `expiry` predicate in queries).
- Config: select the store implementation via the `auth` config block (default to in-memory for single-replica / local, Postgres when configured).

## Acceptance criteria

- [ ] Postgres `NonceStore` implementation with the same `Issue` / `Consume` contract as `InMemory`.
- [ ] Migration registered under `pkg/migrations/apidb/`.
- [ ] `Consume` is atomic and safe across concurrent replicas.
- [ ] Store implementation is selectable via config; in-memory remains the default.
- [ ] Unit tests mirroring `in_memory_test.go` (issue/reuse/consume/expiry/full).
- [ ] Multiple api-server replicas can complete the SIWE login flow across replica boundaries.

## References

- PR #353 — introduces `InMemory` nonce store and the `NonceStore` interface
- `pkg/auth/service/nonce_provider/in_memory.go`
- `pkg/auth/service/login.go`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with pkg/auth/service/nonce_provider/in_memory.go, its interface and in_memory_test.go, then inspect the auth config block and migration patterns under pkg/migrations/apidb/. Implement the Postgres store with the existing Issue/Consume contract, atomic consumption, reuse, expiry, and capacity behavior. Done means the migration and configuration are wired, mirrored tests pass, and cross-replica SIWE login works.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgres
Domain
authentication, backend, databases, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.