GoogleCloudPlatform / GoogleCloudPlatform/pubsec-declarative-toolkit

[BUG] Re-creating the CC GKE cluster after deploying the landing-zone solution requires removal of the requireShieldedVM org policy - kubernetes intent engine will replace it back post-create

Open
#132 20 comments 0 reactions 1 assignee Claimed by @fmichaelobrien View on GitHub
automation bug developer-experience documentation Landing Zone validate_with_v2
Dominant language
Shell
Stars
36
Forks
26
PR merge metrics
No merged PRs in 30d

Description

update: 20231021
Actually for 1 of the 2 - the historical Shielded - we are good with the following override
```
michael@cloudshell:~/kcc-oi/kpt (kcc-oi-9428)$ kubectl get gcp -n policies
NAME AGE READY STATUS STATUS AGE
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/compute-require-shielded-vm-except-mgt-project 9m36s True UpToDate 9m8s
```

see https://github.com/ssc-spc-ccoe-cei/gcp-tools/issues/32

**Describe the bug**
Re-creating the CC GKE cluster after deploying the landing-zone solution requires removal of the requireShieldedVM org policy - kubernetes intent engine will replace it back post-create
After deploying the LZ successfully - deleted the CC cluster, attempt CC re-create - hitting shieldedVM org policy

https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/main/solutions/landing-zone/environments/common/policies/org-policies.yaml#L84

Existing deployment OK
Screen Shot 2022-09-14 at 17 03 42

**To Reproduce**
Steps to reproduce the behavior:
1 - Took a clean organization, installed the CC cluster, deployed the landing-zone solution
2 - deleted the CC cluster
3 - attempt recreation of the CC cluster
4 - run into shieldedVM org policy (previously set by the landing-zone solution)

**Expected behavior**
Deleting the cluster obviously retains org policies - it seems that the landing-zone will disallow a default CC cluster install - which from a security perspective is normal
So the issue is that the org policy must be removed before attempting to deploy the CC cluster again.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Additional context**

Fix is to apply a false flag for the org policy before creating the CC cluster - then the lz solution will return the policy once gke is back up
```
michael@cloudshell:~ (landing-zone-controller-e4g7d)$ gcloud anthos config controller create landing-zone-controller --location northamerica-northeast1 --network kcc-controller --subnet kcc-regional-subnet
Create request issued for: [landing-zone-controller]
Waiting for operation [projects/landing-zone-controller-e4g7d/locations/northamerica-northeast1/operations/operation-1663188232198-5e8a92fc658f2-a614c3a2-993952f2] to complete...failed.
ERROR: (gcloud.anthos.config.controller.create) unexpected error occurred while waiting for SLM operation [projects/krmapihosting-slm/locations/northamerica-northeast1/operations/operation-1663188237478-5e8a93016ea42-84c2a403-f292d464]: errored while waiting for operation: projects/krmapihosting-slm/locations/northamerica-northeast1/operations/operation-1663188237478-5e8a93016ea42-84c2a403-f292d464: Operation failed with error:
generic::invalid_argument: terraform apply failed, error: exit status 1, stderr:
2022/09/14 13:44:18 [DEBUG] Using modified User-Agent: Terraform/0.12.31 Cloud SSA

Error: Error waiting for creating GKE cluster:
(1) Not all instances running in IGM after 17.043475418s. Expected 1, running 0, transitioning 1. Current errors: [CONDITION_NOT_MET]: Instance 'gke-krmapihost-landing-z-default-pool-dbc4594b-xn0b' creation failed: Constraint constraints/compute.requireShieldedVm violated for project projects/landing-zone-controller-e4g7d. Secure Boot is not enabled in the 'shielded_instance_config' field. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information
(2) Not all instances running in IGM after 18.435053483s. Expected 1, running 0, transitioning 1. Current errors: [CONDITION_NOT_MET]: Instance 'gke-krmapihost-landing-z-default-pool-9a70dc96-s8dd' creation failed: Constraint constraints/compute.requireShieldedVm violated for project projects/landing-zone-controller-e4g7d. Secure Boot is not enabled in the 'shielded_instance_config' field. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information
(3) Not all instances running in IGM after 25.990797918s. Expected 1, running 0, transitioning 1. Current errors: [CONDITION_NOT_MET]: Instance 'gke-krmapihost-landing-z-default-pool-a012fae7-q2vd' creation failed: Constraint constraints/compute.requireShieldedVm violated for project projects/landing-zone-controller-e4g7d. Secure Boot is not enabled in the 'shielded_instance_config' field. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information.

on main_gke.tf line 32, in resource "google_container_cluster" "acp_cluster":
32: resource "google_container_cluster" "acp_cluster" {

, stdout:
google_container_cluster.acp_cluster: Creating...
google_container_cluster.acp_cluster: Still creating... [10s elapsed]
google_container_cluster.acp_cluster: Still creating... [20s elapsed]
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.