Azure-Samples / Azure-Samples/streaming-at-scale
Add deduplication in eventhubs-streamanalytics-azuresql
Open
blocked
- Dominant language
- C#
- Stars
- 233
- Forks
- 99
- PR merge metrics
- No merged PRs in 30d
Description
In the solution, generation of duplicate events in locust is disabled (https://github.com/Azure-Samples/streaming-at-scale/commit/23bd9d16fd5da9b3adffdee02103fc77f0e5231d#diff-82da803b8f58c37426d86bd254ace7e4R222) for stream analytics as no mechanism exists to deduplicate. Even now, in case the ASA infra fails and is restarted, duplicates could cause the job to fail since there is a PK on EventId in Azure SQL.
Contributor guide
Assessment
This issue has not been assessed yet.