GoogleCloudPlatform / GoogleCloudPlatform/DataflowTemplates

[Bug]: Datastream-to-Bigquery Dataflow v2 template does not perform merge on CDC data from SQLServer because of incorrect mapping of SQL Server primary key field

Open
#3,805 0 comments 0 reactions 0 assignees View on GitHub
bug p2
Dominant language
Java
Stars
1.3k
Forks
1.1k
Avg merge
2d 20h
Merged PRs (30d)
60

Description

### Related Template(s)

Cloud_Datastream_to_BigQuery

### Template Version

2026-05-12-00_RC00

### What happened?

Cloud_Datastream_to_BigQuery Dataflow v2 template, recently added support for SQLServer CDC data as part of feature request - https://github.com/GoogleCloudPlatform/DataflowTemplates/issues/3410.
This support has missed handling of reading of SQL Server primary keys. SQL Server primary keys have to be read from "replication_index" metadata field instead of the usual "primary_keys" field as is the case with PostgreSQL, MySQL etc.
This does not result in any error, but the Merge Dataset does not have any records populated into them, as there is no Primary key found for the tables.

### Relevant log output

```shell

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.