avniproject / avniproject/snapshot-server
Per-user job state + auto-retry on next scheduler tick
- Dominant language
- JavaScript
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
### Background
Parent: avniproject/avni-client#1942. Depends on the scheduled-generation card. When a user's snapshot fails mid-run, the system must (a) persist that failure visibly and (b) auto-retry on the next scheduler tick with backoff — without requiring an ops endpoint call.
Snapshot-server already has resume-from-failure plumbing: `snapshot_user_job.resume_cursor`, `SyncRunner.resumeContext`, `EntitySyncStatusService.rehydrateFromDb`, and `EntityServiceStub` read-through hydration. Auto-retry should reuse this path so heavy users (~hour-long runs) don't redo committed work on a transient failure. This card wires the scheduler into the existing resume mechanism and adds the missing state-machine + backoff.
### Scope
- Per-user job state in snapshot-server DB with auto-retry, reusing the existing resume cursor.
- Per-user atomicity at page-boundary granularity (existing resume guarantee).
### Acceptance criteria
- [ ] Per-user job table records state (`pending` / `in_progress` / `succeeded` / `failed`), timestamps, attempt count, last error.
- [ ] On scheduler tick, users in `pending` and `failed` are picked up (subject to backoff for `failed`). `in_progress` rows older than a timeout are treated as crashed → eligible to resume from the persisted cursor.
- [ ] Retries reuse the existing `resume_cursor` path — a retried run reopens the partial `.db`, rehydrates `entity_sync_status` + entity cache, and continues from the last committed page boundary. No regression on the existing resume tests.
- [ ] Failure backoff: exponential (e.g. 1h, 4h, 12h, then daily) up to a configurable max attempt count, after which the user stays `failed` until manually retried via the existing restart endpoint.
- [ ] Transient errors (network, S3, sync timeout) increment attempt count and back off via the resume path. Permanent errors (schema mismatch, missing user, 4xx from avni-server) skip retry and stay `failed`.
- [ ] Structured logs surface every state transition with `user`, `org`, `attempt`, `error` for log-based ops monitoring.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.