GoogleCloudPlatform / GoogleCloudPlatform/DataflowTemplates
[Bug]: Cloud_Spanner_to_GCS_Avro template ignores 'tableNames', always exports all tables
- Dominant language
- Java
- Stars
- 1.3k
- Forks
- 1.1k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 60
Description
### Related Template(s)
Cloud_Spanner_to_GCS_Avro
### Template Version
latest
### What happened?
Template ignores `tableNames` parameter.
```sh
gcloud dataflow jobs run spanner-export-job \
--gcs-location=gs://dataflow-templates/latest/Cloud_Spanner_to_GCS_Avro \
--service-account-email=tmp-df-spanner-import@${PROJECT}.iam.gserviceaccount.com \
--project=${PROJECT} \
--region=${REGION} \
--network=${NETWORK} \
--subnetwork=${SUBNETWORK} \
--disable-public-ips \
--parameters="^|^instanceId=${INSTANCE_ID}|databaseId=${DATABASE_ID}|tableNames=table1,table2|shouldExportRelatedTables=false|outputDir=${EXPORT_LOCATION}"
```
Export always contains all tables.
Even if `tableNames` contains a single, independent table.
Export fails (as expected) when nonexistent table is defined.
Resulting export manifest:
```json
{
"tables": [{
"name": "mutex",
"manifestFile": "mutex-manifest.json"
}, {
"name": "ObserverVersion",
"manifestFile": "ObserverVersion-manifest.json"
}, {
"name": "MigrationsHistory",
"manifestFile": "MigrationsHistory-manifest.json"
}, {
"name": "table1",
"manifestFile": "table1-manifest.json"
}, {
"name": "table2",
"manifestFile": "table2-manifest.json"
}],
"databaseOptions": [{
"optionName": "version_retention_period",
"optionType": "STRING",
"optionValue": "1h"
}]
}
```
### Relevant log output
```shell
```
Contributor guide
Assessment
This issue has not been assessed yet.