Azure / Azure/Azure-DataFactory

Azure Data Factory: How to implement change feed in data flow?

Open
#595 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
PowerShell
Stars
529
Forks
623
PR merge metrics
No merged PRs in 30d

Description

I am building a pipeline to move data from CosmosDB to Delta table. ADF supports change feed for CosmosDB by default ONLY IF I select Container as input in the source. Due to requirements I have to use Query as input and as a result I can't select change feed in the source. What's the alternative way to implement Change feed in data flow? I would like to be able to upsert (and possibly delete) records into the delta table.

Currently I am using the Alter row modifier and set Upsert to true() in the alter row conditions section. And this modifier is piped from the source data. Not sure if this is the correct implementation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.