ProjectTech4DevAI / ProjectTech4DevAI/kaapi-backend
Cleanup: Automated data retention system
オープン
@Prajna1999 がすでに取り組んでいます。
2026年9月4日 から。
- 主要言語
- Python
- スター
- 18
- フォーク
- 10
- 平均マージ
- 2日 20時間
- マージ済み PR（30日）
- 14
説明
Is your feature request related to a problem?
We need a system to automatically clean up query and response data in the LLM calls table after a specified retention period. Without this, the database may become cluttered and inefficient.
Describe the solution you'd like
- Implement a cron job to purge query/response data after a configurable retention window (default 1 week).
- Handle one-time backlog separately from regular cron operations.
- Write a cleanup strategy document detailing cron design, batching, and backlog handling.
- Conduct a staging load test by duplicating ~20k LLM call rows ~5x in the
copy_devdatabase to measure cleanup time. Do not use production data. - Ensure S3 cleanup is managed separately and not included in the DB cron/migration code.
Original issue
Context
query + response data in the LLM calls table will be cleaned up via a cron job after a configurable retention window (starting with 1 week).
A step-by-step cleanup strategy doc is to be written (cron design, batching, one-time backlog handling).
Scope / Acceptance criteria
- Cron job to purge query/response data in the LLM calls table after a configurable retention window (default 1 week).
- Handle one-time backlog separately from steady-state cron.
- Write cleanup strategy doc (cron design, batching, backlog).
- Staging load test: reuse the empty
copy_devdatabase, duplicate staging's ~20k LLM call rows ~5x to reach ~1 lakh, and measure cleanup time. Do NOT pull prod data. - Keep S3 cleanup separate — do not bundle S3 deletion into the DB cron/migration code.
Notes
- Guardrails input/output data is retained separately (anonymized, tagged as product enhancement) and is NOT part of this cleanup.
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
評価
この issue はまだ評価されていません。