Cloud-CV / Cloud-CV/EvalAI

[Backend] Add feature to stop/delete code upload infrastructure

Open
#3,231 6 comments 0 reactions 0 assignees View on GitHub
backend GSOC-2021 hard-to-fix
Dominant language
Python
Stars
2k
Forks
984
Avg merge
2h 54m
Merged PRs (30d)
14

Description

### Description

We have an automated set up to create a code upload worker cluster in host's AWS account which creates all the required services for code upload challenge submission evaluation. We want to add automated support to stop/delete the code upload worker infrastructure. This requires -

- [ ] Delete code upload worker running on EvalAI fargate
- [ ] Deleting EKS cluster nodegroup
- [ ] Deleting EKS cluster
- [ ] Deleting cloudwatch log groups
- [ ] Deleting the VPC and associated subnets, security group and route table
- [ ] Deleting IAM roles

Metadata of mentioned service is stored in `ChallengeEvaluationCluster` model, you can find the code [here](https://github.com/Cloud-CV/EvalAI/blob/master/apps/challenges/models.py#L523). Codebase for starting these services is available [here](https://github.com/Cloud-CV/EvalAI/blob/master/apps/challenges/aws_utils.py#L1110).

The approach for this feature is to:

- [ ] Add a django admin action for challenge model which will trigger resource deletion
- [ ] Create multiple celery tasks which will handle deletion of each service
- [ ] Trigger the service deletion using boto3 APIs and to get the resource ids use `ChallengeEvaluationCluster` model
- [ ] Send an email to challenge host when all services are stopped successfully

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.