redpanda-data / redpanda-data/connect
microsoft_sql_server_cdc: use cluster index when primary key is missing
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 8.8k
- Forks
- 969
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 64
Description
Hi,
When using snapshot we can use cluster index instead a primary key.
We can try additional query here to search cluster index.
As alternative, to add config field snapshot_order_columns that defines order by columns.
To find cluster index:
SELECT t.name, c.name FROM sys.indexes AS i
JOIN sys.index_columns ic ON i.object_id = ic.object_id AND i.index_id = ic.index_id
JOIN sys.columns c ON ic.object_id = c.object_id AND ic.column_id = c.column_id
JOIN sys.tables t ON i.object_id = t.object_id
JOIN sys.schemas s ON t.schema_id = s.schema_id
WHERE i.type = 1 -- 1 is cluster
AND s.name = ? -- schema
AND t.name = ? -- table
ORDER BY ic.key_ordinal;
Regards
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in internal/impl/mssqlserver/replication/snapshot.go at the linked lookup around line 251, and trace how snapshot ordering is chosen when a primary key is absent. Use the proposed sys.indexes query as the reference for finding clustered-index columns. Done means snapshots can use the clustered index when no primary key exists.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- data-engineering, databases, stream-processing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100