kubernetes: build with an initContainer or a Job
- Langage dominant
- Go
- Étoiles
- 300
- Forks
- 64
- Merge moyen
- 20 min
- PR mergées (30 j)
- 1
Description
## Context
Currently, envbuilder runs at the start up of a workspace, exposing elements of the buildtime to the runtime and vice-versa:
* Build secrets (e.g. dockerconfig)
* Environment variables (#91)
* Mounts (#187)
* Privileges (#181)
* Container layers are downloaded in each container rather than on the nodes
* ... (feel free to grow the list)
## Proposal 1: initContainer
1. Envbuilder would build the image as an [initContainer](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) and push it to a container registry
2. The main container would pull and run the image (todo: validate that the pod can be created without the image existing yet)
This would require to generate/know the image reference ahead of time.
## Proposal 2: Kubernetes Job
Entire decoupling of buildtime and runtime:
1. Envbuilder runs as Kubernetes `Job` to build and push the container image
2. It writes a `ConfigMap` with the reference of the built image
3. Terraform [waits for completion](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/job#wait_for_completion) of the `Job`
4. Terraform reads the `ConfigMap` with [`kubernetes_config_map`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/data-sources/config_map) datasource ([explicitly depending on](https://developer.hashicorp.com/terraform/language/meta-arguments/depends_on) the `Job` creation)
5. The image reference from the `ConfigMap` is then used to create a `Deployment`
6. A short [`ttl_seconds_after_finished`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/job#ttl_seconds_after_finished) would allow clean up of the Job for it to be recreated on the next `terraform apply`
The `ConfigMap` could be used to share of information between `envbuilder` and Terraform (#121), like the volumes defined in the `devcontainer.json` (#220)
Detail to consider: I believe the Coder server starts streaming the logs from the deployment after the `terraform apply` has finished, it would need to be able to do it for the `Job` while the apply is running to expose the build logs to the user.
Is it something that has been thought of/done but not documented yet?
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Piste de recherche
No files or tests are named. Start by reviewing the current Kubernetes deployment and Terraform resources, then trace how envbuilder builds during workspace startup; done requires choosing and implementing one proposal, validating the image-reference handoff, and addressing build-log streaming.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- kubernetes, terraform
- Domaine
- cloud, infrastructure
- Type d'issue
- Fonctionnalité
- Difficulté
- 5/5
- Temps estimé
- Plus d'une semaine
- Activité
- À l'abandon
- Clarté
- À clarifier
- Accessibilité débutants
- 25/100