google / google/stellar-engine
[Bug] Standard Brownfield auto-discovers a nonexistent CMEK keyring (${Cap_Env}-${Tenant}-keyring) with no override prompt, writing a broken kms_key_id that fails the gemini-stage-0 apply
- Dominant language
- HCL
- Stars
- 49
- Forks
- 20
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 28
Description
## Bug Description
In `gemini-enterprise/deploy.sh`, standard Brownfield (option 2) derives the CMEK keyring name as `${Cap_Env}-${Tenant}-keyring` (e.g. `Prod--keyring`) in the tenant iac-core project and looks up the Gemini resources key there (`US_KEYRING_NAME` set ~Line 618). On a Stellar Engine landing zone the key actually lives in the `prod-us` keyring under `-prod-sec-core-0`, not the derived keyring, so `keyrings describe` misses it, deploy.sh prints "US Keyring not found" and leaves `CMEK_US_RESOURCES_KEY` blank (~Lines 639–641). The three interactive CMEK override prompts (`Enter CMEK Project ID` / `US Multi-Region Keyring ID` / `US Gemini Resources Key ID`) exist only inside the `elif IS_CUSTOM` (Custom Brownfield, option 3) branch (~Lines 675, L699–706), so standard Brownfield gives the operator no way to correct the value. With the resources key blank, the data-store step later auto-creates the key on the wrong keyring (`if [[ -z "$CMEK_US_RESOURCES_KEY" ]]`, ~Line 1937): `kms keys create` returns `NOT_FOUND`, the Discovery Engine service-agent grant fails, cmekConfig registration returns HTTP 400, and the generated `gemini-stage-0/terraform.tfvars` carries the wrong `kms_key_id`, so the subsequent automatic `terraform apply` fails.
## Environment and Deployment Context
* **Stellar Engine Version/Commit:** `main` @ `3728fc98`
* **Deployment Type:**
* [ ] US Region Restricted (e.g., Access Policy constraint)
* [ ] FedRAMP Medium
* [x] FedRAMP High
* [ ] DoD IL4
* [ ] DoD IL5
* [ ] Stand-alone / Custom
* **Affected Component:** `blueprints/fedramp-high/gemini-enterprise/deploy.sh` — `discover_infrastructure()` CMEK block: keyring derive (~Line 618), "US Keyring not found" blank (~Lines 639–641), Custom-only override prompts (~Lines 675, L699–706), data-store auto-create (~Line 1937), cmekConfig registration (~Lines 1960–1989)
* **Terraform Version:** deploy.sh pins Terraform 1.12.2 (tfenv)
## Steps to Reproduce
1. On a healthy SE FRH landing zone, from `---main-0`, run `./deploy.sh`; choose Brownfield (2), then Step 1.
2. When prompted, enable CMEK for the data store.
3. Observe deploy.sh derive keyring `--keyring` in the iac-core project, print "US Keyring not found", and leave the resources key blank — no override prompt appears (those are Custom-Brownfield only).
4. deploy.sh auto-creates the key on the wrong keyring → `NOT_FOUND`; the DE service-agent grant fails; cmekConfig registration returns 400; the wrong `kms_key_id` is written to `terraform.tfvars`; the automatic apply then fails on it.
## Expected Behavior
Standard Brownfield should discover the real SE key (`projects/-prod-sec-core-0/locations/us/keyRings/prod-us/cryptoKeys/gemini-enterprise`) — or expose the CMEK override prompts in standard Brownfield — so the data store is encrypted with the sovereign key without hand-editing tfvars.
## Actual Behavior
On a fresh SE tenant the CMEK step silently targets a nonexistent keyring, the data store cannot be CMEK-encrypted, and the apply fails until the operator manually corrects `gemini-stage-0/terraform.tfvars`.
## Relevant Logs and Errors
```
CMEK for Data Stores requested. Ensuring key exists...
Target Project: -prod--iac-core-0
Keyring: Prod--keyring
Creating Key 'gemini-enterprise'...
ERROR: (gcloud.kms.keys.create) NOT_FOUND: KeyRing projects/.../keyRings/Prod--keyring not found.
WARNING: Failed to grant IAM binding to Discovery Engine service account.
You might need 'roles/cloudkms.admin' on the key project.
Failed to register CMEK key. HTTP Status: 400 (FAILED_PRECONDITION)
```
## Additional Context
Distinct from #106. Same deploy.sh surface as #114.
Contributor guide
Research direction
Start in blueprints/fedramp-high/gemini-enterprise/deploy.sh, reading discover_infrastructure() around the CMEK discovery and override logic, then trace the data-store auto-create and cmekConfig registration sections. Reproduce option 2, Step 1 with CMEK enabled and verify that the real sovereign key or an operator override is used, the generated gemini-stage-0/terraform.tfvars has the correct kms_key_id, and the automatic apply succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, shell, terraform
- Domain
- cloud, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100