Azure / Azure/Azure-DataFactory

Change Data Capture tracking deletes from SQL Server does not work with Delta sink

Open
#531 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
PowerShell
Stars
529
Forks
623
PR merge metrics
No merged PRs in 30d

Description

Hi,
I am testing the new [change data capture](https://learn.microsoft.com/en-us/azure/data-factory/concepts-change-data-capture) feature. It seems that when rows are deleted from my Azure SQL Server source, they are not deleted from the destination delta table, but instead, the row is appended again. I appreciate you may not want to run an actual DELETE operation on the delta table due to the associated overhead, but I would at least expect a boolean _soft-deleted_ flag so we know how to treat this downstream.
Hope that makes sense.

edit: to clarify I am using the Change Data Capture resource rather than the mapping data flow.

update: this issue also occurs with a flat file destination

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.