Azure / Azure/Azure-DataFactory

Copy Activity: Copying data from and to Azure Cosmos DB for MongoDB: Failing due to incorrect Data Type

Open
#524 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
PowerShell
Stars
529
Forks
623
PR merge metrics
No merged PRs in 30d

Description

I am using Azure Data Factory `Copy Activity` to copy a collection from and to `Azure Cosmos DB for MongoDB`.

The data that is getting saved in the `from` collection, we don't have control over and is managed by a third party.

This particular collection has a property, for some documents, it has `Int64` values, and for some documents, it's `string` values (for some odd reason). And `Copy Activity` is failing due to this. I can get the action passed by setting `Fault tolerance = Skip incompatible rows`, but then documents with mismatching schema are getting skipped.

But I would expect `Copy Activity` to copy the data over regardless of schema validation, especially when it's NoSQL-type data.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.