cupOJoseph / cupOJoseph/spaceCanvas

Implement chunked Travis household and turf materialization

Open
#2 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Problem
The production `registered_voter` table has 929,802 Travis County rows. The current `reset_demo_data` reducer tries to scan and materialize all household knock targets and turfs in one transaction. On Maincloud this overloaded the instance and returned HTTP 530.

## Goal
Implement a resumable chunked materialization flow that derives household-level `voter` knock targets, `turf`, and `turf_stats` rows from `registered_voter` without overloading SpacetimeDB.

## Suggested Approach
- Add materialization progress state, cursor, and phase tracking.
- Process `registered_voter` in bounded batches.
- Group voters by normalized household key over multiple reducer calls.
- Generate roughly 200-registered-voter turf chunks.
- Make the flow idempotent and resumable after interruption.
- Add a CLI/script command to run the chunks until complete.

## Acceptance Criteria
- Running the materialization against `spacecanvas-5rvak` completes without server fatal errors.
- `registered_voter` remains preserved.
- Derived `voter` rows represent households with `registered_voter_count`.
- `turf_stats.total_voters` sums to the registered voters represented by derived household rows.
- A verification script reports row counts and progress.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.