Azure-Samples / Azure-Samples/streaming-at-scale

Add deduplication in eventhubs-streamanalytics-azuresql

Open
#44 0 comments 0 reactions 0 assignees View on GitHub
blocked
Dominant language
C#
Stars
233
Forks
99
PR merge metrics
No merged PRs in 30d

Description

In the solution, generation of duplicate events in locust is disabled (https://github.com/Azure-Samples/streaming-at-scale/commit/23bd9d16fd5da9b3adffdee02103fc77f0e5231d#diff-82da803b8f58c37426d86bd254ace7e4R222) for stream analytics as no mechanism exists to deduplicate. Even now, in case the ASA infra fails and is restarted, duplicates could cause the job to fail since there is a PK on EventId in Azure SQL.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.