Kong / Kong/kubernetes-testing-framework

Make GKE clusters ephemeral by default

Open
#730 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
44
Forks
11
PR merge metrics
No merged PRs in 30d

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Problem Statement

Use of KTF outside CI may leak GKE clusters due to humans forgetting to delete them after usage. To avoid unnecessary cloud spend, we'd like a means to have them default to ephemeral unless marked otherwise.

### Proposed Solution

- Add a GKE cluster label indicating a deletion date.
- Default this label to three days after creation.
- Provide a flag allowing users to override the default.
- Add a scheduled task to delete clusters past their expiration.

### Additional information

Elsewhere we run cleanup jobs as part of CI in projects that use KTF. There isn't really a great CI location for _this_ scheduled deletion, since this is intended for clusters created outside CI. I don't think it makes sense to run this in the KTF repo itself.

GCP does provide https://cloud.google.com/scheduler with 3 free jobs a month ($0.10/job/month after). We should use it as a project-agnostic deletion method.

We may want a "never" option but it's simpler to just use dates as label values, and we should probably discourage indefinite lifetime clusters anyway. Setting the duration to 9999 or something similarly ridiculous or manually removing the label should be sufficient if you really, really want to avoid the cleanup job.

### Acceptance Criteria

- [ ] KTF's GKE cluster creation utility provides a CLI flag that takes a number of days as input and sets an expiration date label.
- [ ] The CLI flag defaults to 3 days.
- [ ] We provide an example GKE Cloud Scheduler configuration that deletes clusters past expiration.
- [ ] We use the scheduler job on our GKE project.

Contributor guide

Open the contributing guide

Research direction

Start at KTF's GKE cluster creation utility and trace how its CLI options and cluster labels are configured. Define the expiration-date label and three-day default, then document the CLI override and provide the requested GKE Cloud Scheduler configuration. Done includes applying the scheduler job to the GKE project and covering all listed acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, go, kubernetes
Domain
cloud, devops, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.