google / google/stellar-engine

[Bug] Standard Brownfield auto-discovers a nonexistent CMEK keyring (${Cap_Env}-${Tenant}-keyring) with no override prompt, writing a broken kms_key_id that fails the gemini-stage-0 apply

Open
#132 0 comments 0 reactions 0 assignees View on GitHub
bug gemini for government Level of Effort - Low Priority - Medium
Dominant language
HCL
Stars
49
Forks
20
Avg merge
3d 14h
Merged PRs (30d)
28

Description

## Bug Description
In `gemini-enterprise/deploy.sh`, standard Brownfield (option 2) derives the CMEK keyring name as `${Cap_Env}-${Tenant}-keyring` (e.g. `Prod--keyring`) in the tenant iac-core project and looks up the Gemini resources key there (`US_KEYRING_NAME` set ~Line 618). On a Stellar Engine landing zone the key actually lives in the `prod-us` keyring under `-prod-sec-core-0`, not the derived keyring, so `keyrings describe` misses it, deploy.sh prints "US Keyring not found" and leaves `CMEK_US_RESOURCES_KEY` blank (~Lines 639–641). The three interactive CMEK override prompts (`Enter CMEK Project ID` / `US Multi-Region Keyring ID` / `US Gemini Resources Key ID`) exist only inside the `elif IS_CUSTOM` (Custom Brownfield, option 3) branch (~Lines 675, L699–706), so standard Brownfield gives the operator no way to correct the value. With the resources key blank, the data-store step later auto-creates the key on the wrong keyring (`if [[ -z "$CMEK_US_RESOURCES_KEY" ]]`, ~Line 1937): `kms keys create` returns `NOT_FOUND`, the Discovery Engine service-agent grant fails, cmekConfig registration returns HTTP 400, and the generated `gemini-stage-0/terraform.tfvars` carries the wrong `kms_key_id`, so the subsequent automatic `terraform apply` fails.

## Environment and Deployment Context
* **Stellar Engine Version/Commit:** `main` @ `3728fc98`
* **Deployment Type:**
* [ ] US Region Restricted (e.g., Access Policy constraint)
* [ ] FedRAMP Medium
* [x] FedRAMP High
* [ ] DoD IL4
* [ ] DoD IL5
* [ ] Stand-alone / Custom
* **Affected Component:** `blueprints/fedramp-high/gemini-enterprise/deploy.sh` — `discover_infrastructure()` CMEK block: keyring derive (~Line 618), "US Keyring not found" blank (~Lines 639–641), Custom-only override prompts (~Lines 675, L699–706), data-store auto-create (~Line 1937), cmekConfig registration (~Lines 1960–1989)
* **Terraform Version:** deploy.sh pins Terraform 1.12.2 (tfenv)

## Steps to Reproduce
1. On a healthy SE FRH landing zone, from `---main-0`, run `./deploy.sh`; choose Brownfield (2), then Step 1.
2. When prompted, enable CMEK for the data store.
3. Observe deploy.sh derive keyring `--keyring` in the iac-core project, print "US Keyring not found", and leave the resources key blank — no override prompt appears (those are Custom-Brownfield only).
4. deploy.sh auto-creates the key on the wrong keyring → `NOT_FOUND`; the DE service-agent grant fails; cmekConfig registration returns 400; the wrong `kms_key_id` is written to `terraform.tfvars`; the automatic apply then fails on it.

## Expected Behavior
Standard Brownfield should discover the real SE key (`projects/-prod-sec-core-0/locations/us/keyRings/prod-us/cryptoKeys/gemini-enterprise`) — or expose the CMEK override prompts in standard Brownfield — so the data store is encrypted with the sovereign key without hand-editing tfvars.

## Actual Behavior
On a fresh SE tenant the CMEK step silently targets a nonexistent keyring, the data store cannot be CMEK-encrypted, and the apply fails until the operator manually corrects `gemini-stage-0/terraform.tfvars`.

## Relevant Logs and Errors
```
CMEK for Data Stores requested. Ensuring key exists...
Target Project: -prod--iac-core-0
Keyring: Prod--keyring
Creating Key 'gemini-enterprise'...
ERROR: (gcloud.kms.keys.create) NOT_FOUND: KeyRing projects/.../keyRings/Prod--keyring not found.
WARNING: Failed to grant IAM binding to Discovery Engine service account.
You might need 'roles/cloudkms.admin' on the key project.
Failed to register CMEK key. HTTP Status: 400 (FAILED_PRECONDITION)
```

## Additional Context
Distinct from #106. Same deploy.sh surface as #114.

Contributor guide

Open the contributing guide

Research direction

Start in blueprints/fedramp-high/gemini-enterprise/deploy.sh, reading discover_infrastructure() around the CMEK discovery and override logic, then trace the data-store auto-create and cmekConfig registration sections. Reproduce option 2, Step 1 with CMEK enabled and verify that the real sovereign key or an operator override is used, the generated gemini-stage-0/terraform.tfvars has the correct kms_key_id, and the automatic apply succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, shell, terraform
Domain
cloud, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.