Automated housekeeping

Why are we doing this?

There are some tasks it's useful to run periodically in production. We should automate these.

• Vacuuming a local copy of the core DB reduced its size by 15% (Slack thread) so doing so on the real server could reduce space used by the DB and its backups.
• Vacuuming coding system databases on creation https://github.com/opensafely-core/opencodelists/issues/2383
• There are thousands of expired Django sessions in the core DB (Slack thread). These don't take up a huge amount of space but it does gradually increase over time and is multiplied by the number of backups. (See Job Server clearsessions job).
• Applying Django management command migrate --prune removes record of stale migrations from the DB and avoids any clashes after squashing migrations.

How will we know when it's done?

A cron or other mechanism for periodically automatically executing the tasks is operational.

What are we doing?

• Creating an automated housekeeping task that periodically (weekly? at night and not clashing with others such as backups?) addresses the above issues.
• Linking it to Sentry?
• Check whether any of these considerations apply in Job Server and if so raise an issue there.

Defining delivery tasks guidance

Start by reviewing the Job Server clearsessions job and the Django management commands named in the issue, then examine how production database tasks and backups are scheduled. Clarify the mechanism, timing, Sentry integration, and whether Job Server needs a separate issue. Done means a periodic operational task runs the agreed housekeeping actions.