[Backend] Add feature to stop/delete code upload infrastructure
- Dominant language
- Python
- Stars
- 2k
- Forks
- 984
- Avg merge
- 2h 54m
- Merged PRs (30d)
- 14
Description
### Description
We have an automated set up to create a code upload worker cluster in host's AWS account which creates all the required services for code upload challenge submission evaluation. We want to add automated support to stop/delete the code upload worker infrastructure. This requires -
- [ ] Delete code upload worker running on EvalAI fargate
- [ ] Deleting EKS cluster nodegroup
- [ ] Deleting EKS cluster
- [ ] Deleting cloudwatch log groups
- [ ] Deleting the VPC and associated subnets, security group and route table
- [ ] Deleting IAM roles
Metadata of mentioned service is stored in `ChallengeEvaluationCluster` model, you can find the code [here](https://github.com/Cloud-CV/EvalAI/blob/master/apps/challenges/models.py#L523). Codebase for starting these services is available [here](https://github.com/Cloud-CV/EvalAI/blob/master/apps/challenges/aws_utils.py#L1110).
The approach for this feature is to:
- [ ] Add a django admin action for challenge model which will trigger resource deletion
- [ ] Create multiple celery tasks which will handle deletion of each service
- [ ] Trigger the service deletion using boto3 APIs and to get the resource ids use `ChallengeEvaluationCluster` model
- [ ] Send an email to challenge host when all services are stopped successfully
Contributor guide
Assessment
This issue has not been assessed yet.