developmentseed / developmentseed/pearl-backend
Deal with azurerm updates
- Dominant language
- JavaScript
- Stars
- 58
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
Recently, when deploying an unrelated change, the deploy process failed because of updates to the `azurerm` provider that caused things to break. One, was a deprecation to the `addon_profile` property for the `azurerm` `kubernetes_cluster` resource, and one was a change in default properties for the Public IP resource.
Azure does not deal very well with updating properties in place, and in both these cases, would try to delete and re-create these resources for ANY change to its properties. For now, have set the lifecycle property for the cluster as well as the IP address to ignore any changes: https://github.com/developmentseed/pearl-backend/blob/develop/deployment/terraform/resources/aks.tf#L2
For the Public IP address, I feel setting the lifcycle property to always ignore changes is fine - I'm okay treating the Public IP address as immutable after creation - we can add a comment in the file that if one needs to change properties of the public IP address, one just needs to do it in the azure console and replicate the same changes in the tf file.
For the AKS resource, this is not fine - not managing the cluster via terraform after initial creation sounds like it would cause a lot of headache down the road. The current problem with terraform re-creating the cluster is:
- Fear of the unknown: we're not fully sure what might break if the entire cluster is re-created. This **should** be fine, but is something we should perhaps test separately.
- Practically, terraform currently fails because of the way we are passing credentials of the cluster to the Helm terraform provider. Essentially, when using a `kubernetes` or `helm` provider, those terraform resources should not be applied in the same terraform module as is creating the `kubernetes_cluster` resource. This is the Warning on this page: https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs . The suggested fix is to perform the terraform apply in two steps - the first applies just the cluster resource, and the second applies the kubernetes / helm provider that depends on values outputted by the cluster resource for auth. So, we should figure out testing splitting our `terraform apply` into two steps like in the example here: https://github.com/hashicorp/terraform-provider-kubernetes/blob/main/_examples/aks/README.md
So, there's a few Next Actions here:
- [x] Pin the version of `azurerm` we are using so that these issues don't bite us randomly / we're a bit more in control of upgrade paths.
- [ ] Add a comment to the Public IP resource indicating it is immutable after creation.
- [ ] Split up the terraform apply step as per above.
- [ ] Remove the `lifecycle_changes` argument to the `azurerm_kubernetes_cluster` resource and test that it all works.
This is all a bit non-ideal - I think we missed this because while the big yellow Warning exists on the docs for the `kubernetes` provider, it does not exist on the docs for the `helm` provider here: https://registry.terraform.io/providers/hashicorp/helm/latest/docs#in-cluster-config and this behaviour is definitely non-intuitive.
@geohacker I can probably work through these.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.