learningequality / learningequality/studio
Set per-table autovacuum and analyze thresholds on large tables
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 191
- Forks
- 307
- Avg merge
- 5d 6h
- Merged PRs (30d)
- 10
Description
❌ This issue is not open for contribution. Visit Contributing guidelines to learn about the contributing process and how to find suitable issues.
Overview
- Default
autovacuum_vacuum_scale_factor = 0.2defers vacuum until 20% of a table's rows are dead — 3.36M tuples oncontentnode, 11.9M onassessmentitem. - Neither table has ever been vacuumed or analyzed, leaving the planner without statistics for the largest table in the database.
Complexity: Low
Target branch: hotfixes
Context
ALTER TABLE ... SET (autovacuum_*)is metadata-only and does not rewrite the table, but takes a briefACCESS EXCLUSIVElock. Oncontentnodethat can queue behind a long-running transaction and block everything behind it — gunicorn's timeout is 4000s, so long transactions are possible.pg_class.reloptionsis currently null for all three tables, and Django exposes no model-level API for storage parameters.- The app image has no
psql, soANALYZEmust be issued through a database cursor.
The Change
- A migration should set per-table autovacuum and analyze thresholds on
contentnode,assessmentitem, andfile:autovacuum_vacuum_scale_factor = 0.01andautovacuum_analyze_scale_factor = 0.005. fileshould additionally getautovacuum_vacuum_insert_scale_factor = 0.05— at 113M insert-heavy rows the dead-tuple threshold alone never fires, but the visibility map still needs maintaining.- The migration should set a
lock_timeoutand fail rather than retry, so a blockedACCESS EXCLUSIVEacquisition does not queue readers behind repeated attempts. - A general-purpose management command should run
ANALYZEagainst tables named at invocation, issued through a database cursor. make deploy-migrateshould invoke that command for the three tables, per the procedure at Makefile:32-40.
Acceptance Criteria
General
-
pg_class.reloptionsoncontentcuration_contentnode,contentcuration_assessmentitem, andcontentcuration_filecontainsautovacuum_vacuum_scale_factor=0.01andautovacuum_analyze_scale_factor=0.005 -
contentcuration_fileadditionally hasautovacuum_vacuum_insert_scale_factor=0.05 - The migration reverses cleanly, resetting all three tables to no storage parameters
- The migration aborts with a non-zero exit when it cannot acquire its lock within
lock_timeout - A management command runs
ANALYZEagainst tables passed as arguments -
make deploy-migrateinvokes that command for the three tables
Testing
-
last_analyzeis non-null on all three tables after the deploy step runs -
last_autovacuumbecomes non-null oncontentnodewithin 24 hours of the thresholds applying - Unit test covers the command's table-argument handling and its error on an unknown table
AI usage
Drafted with Claude Code, which measured the vacuum state, analyze state, and row counts cited here through read-only queries against the production database. I confirmed the deploy-migrate procedure and migration precedent against the Studio source, and set the threshold values and the fail-loudly behaviour.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the migration conventions and the deploy procedure at Makefile:32-40, then locate the management-command entry point used for database operations. Implement the reversible table settings, lock-timeout failure behavior, and table-argument validation described here. Done means the migration, ANALYZE command, deploy invocation, and listed tests satisfy the acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, postgresql, python
- Domain
- backend, databases, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100