cockroachdb / cockroachdb/cockroach

drt: fetching datadog-api-key is broken

Open
#167,080 1 comment 0 reactions 0 assignees View on GitHub
A-testeng-foundations branch-master C-bug T-testeng
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Several drt configs rely on `pkg/cmd/drtprod/scripts/setup_datadog_cluster` which retrieves the datadog API key from the secrets vault. This used to work previously. However, now IAM permissions for `Editor` do not have the required `secretmanager.versions.access`, resulting in the following failure,

```
gcloud secrets versions access latest --secret datadog-api-key
ERROR: (gcloud.secrets.versions.access) PERMISSION_DENIED: Permission 'secretmanager.versions.access' denied on resource (or it may not exist). This command is authenticated as [622274581499-compute@developer.gserviceaccount.com](mailto:622274581499-compute@developer.gserviceaccount.com) which is the active account specified by the [core/account] property.

'@type': type.googleapis.com/google.rpc.ErrorInfo
domain: iam.googleapis.com
metadata:
permission: secretmanager.versions.access
reason: IAM_PERMISSION_DENIED
```

This means that `roachtest_operations_run.sh` fails on the workload node, since it doesn't have the permission to fetch the API key.

```
grep -r "secrets versions" pkg/cmd/drt/scripts
pkg/cmd/drt/scripts/roachtest_operations_run.sh: if ! secret_value="$(gcloud --project=cockroach-drt secrets versions access latest \
```

Jira issue: CRDB-62237

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.