🗂️ Implement Submission & Logs Retention Policy Post-Challenge (1-Month Retention)
- Dominant language
- Python
- Stars
- 2k
- Forks
- 984
- Avg merge
- 2h 54m
- Merged PRs (30d)
- 14
Description
**Description:**
To improve data storage efficiency and reduce long-term infrastructure costs, we propose implementing a **retention policy** for **submission files and logs** after a challenge has ended on EvalAI.
### 🎯 Objective
Once a challenge has ended, **submission metadata should be preserved**, but the following associated artifacts should be **retained for only 1 month**:
* Submission input files (e.g., prediction files, zip uploads)
* Submission output files (e.g., logs, result archives, stdout/stderr logs)
After the 1-month grace period, the storage used by these large artifacts should be released automatically.
---
### 🛠️ Proposed Implementation
#### 1. **Retention Trigger**
* Trigger based on the `end_date` of the challenge phase.
* Start countdown only if the `challenge_phase.is_public` is `False` (i.e., not accepting any more submissions).
#### 2. **Scheduled Cleanup**
* A periodic background job (e.g., via Celery beat or AWS Lambda/CloudWatch Events) that:
* Identifies expired submissions.
* Deletes associated large files from storage (AWS S3 or others).
* Keeps submission records in the database for audit/reference.
#### 3. **Audit Trail / Flags**
* Add a flag like `is_artifact_deleted` to each submission to avoid repeated deletion attempts.
* Log the deletion with timestamps for traceability.
#### 4. **Notification (Optional Enhancement)**
* Notify challenge hosts 14 days before deletion starts via email.
* Allow manual override if a host wants to retain data longer.
---
### 🔐 Considerations
* Ensure challenge host consent is accounted for — either as a configurable retention policy per challenge or platform-wide with clear documentation.
* Submissions from **host users** or **evaluation script outputs** may need a different policy if used for benchmarking or demo purposes.
---
### 📌 Related Work / References
Ref: https://github.com/Cloud-CV/EvalAI/pull/4586
Contributor guide
Assessment
This issue has not been assessed yet.