GoogleCloudPlatform / GoogleCloudPlatform/DataflowTemplates

[Bug]: Cloud_Spanner_to_GCS_Avro template ignores 'tableNames', always exports all tables

Open
#3,917 4 comments 0 reactions 0 assignees View on GitHub
bug p2
Dominant language
Java
Stars
1.3k
Forks
1.1k
Avg merge
2d 20h
Merged PRs (30d)
60

Description

### Related Template(s)

Cloud_Spanner_to_GCS_Avro

### Template Version

latest

### What happened?

Template ignores `tableNames` parameter.

```sh
gcloud dataflow jobs run spanner-export-job \
--gcs-location=gs://dataflow-templates/latest/Cloud_Spanner_to_GCS_Avro \
--service-account-email=tmp-df-spanner-import@${PROJECT}.iam.gserviceaccount.com \
--project=${PROJECT} \
--region=${REGION} \
--network=${NETWORK} \
--subnetwork=${SUBNETWORK} \
--disable-public-ips \
--parameters="^|^instanceId=${INSTANCE_ID}|databaseId=${DATABASE_ID}|tableNames=table1,table2|shouldExportRelatedTables=false|outputDir=${EXPORT_LOCATION}"
```

Export always contains all tables.
Even if `tableNames` contains a single, independent table.

Export fails (as expected) when nonexistent table is defined.

Resulting export manifest:
```json
{
"tables": [{
"name": "mutex",
"manifestFile": "mutex-manifest.json"
}, {
"name": "ObserverVersion",
"manifestFile": "ObserverVersion-manifest.json"
}, {
"name": "MigrationsHistory",
"manifestFile": "MigrationsHistory-manifest.json"
}, {
"name": "table1",
"manifestFile": "table1-manifest.json"
}, {
"name": "table2",
"manifestFile": "table2-manifest.json"
}],
"databaseOptions": [{
"optionName": "version_retention_period",
"optionType": "STRING",
"optionValue": "1h"
}]
}
```

### Relevant log output

```shell

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.