GoogleCloudPlatform / GoogleCloudPlatform/deploymentmanager-samples
BigQuery log export
- Dominant language
- Jinja
- Stars
- 951
- Forks
- 700
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I'm trying to export some logs to bigquery with the following configuration:
```
- name: BQ_LOG-bigquery-dataset
type: gcp-types/bigquery-v2:datasets
properties:
access:
- role: OWNER
specialGroup: projectOwners
datasetReference:
datasetId: BQ_LOG
projectId: my-project
description: Dataset BigQuery Logs Export
friendlyName: Dataset BigQuery Logs Export
location: EU
- name: log-sink-BQ_LOG-bigquery
type: gcp-types/logging-v2:projects.sinks
properties:
destination: bigquery.googleapis.com/projects/my-project/datasets/$(ref.BQ_LOG-bigquery-dataset.datasetReference.datasetId)
filter: resource.type="bigquery_resource"
project: my-project
sink: log-sink-BQ_LOG-bigquery
uniqueWriterIdentity: true
- name: BQ_LOG-bigquery-dataset-set-sink-iam
action: gcp-types/bigquery-v2:bigquery.datasets.patch
properties:
accesses:
- role: OWNER
specialGroup: projectOwners
- role: WRITER
userByEmail: $(ref.log-sink-BQ_LOG-bigquery.writerIdentity)
datasetId: $(ref.BQ_LOG-bigquery-dataset.datasetReference.datasetId)
projectId: my-project
```
What happen is:
- the dataset BQ_LOG is created
- the sink log-sink-BQ_LOG-bigquery is created
- I've no evidence in Stackdriver Logging about bigquery.datasets.patch action
- the dataset BQ_LOG at the end has only the **projectOwners:OWNER** access
- the DM finish successfully, but as I've said the deployment doesn't make what I want
I've found that the issue is _accesses_ and the right config is _access_ , but no error is fired!
With the correct configuration instead:
- the following error is fired: Email address contains invalid user name
- the _translated_ configuration for BQ_LOG-bigquery-dataset-set-sink-iam resource has **userByEmail: serviceAccount:p770900990982-803178@gcp-sa-logging.iam.gserviceaccount.com**
I don't know how to take a substring of a referenced value.
Do you know how can I achieve the target?
Thanks
Contributor guide
Assessment
This issue has not been assessed yet.