Azure-Samples / Azure-Samples/azure-sql-db-change-stream-debezium
history eventhub is configured with --cleanup-policy Delete instead of Compact.
- Dominant language
- C#
- Stars
- 105
- Forks
- 37
- PR merge metrics
- No merged PRs in 30d
Description
The sample history hub configured is configured as follows.
```
echo "deploying schema history event hub"
az eventhubs eventhub create \
--resource-group $RESOURCE_GROUP \
--namespace-name $EVENTHUB_NAMESPACE \
--name $EVENTHUB_SCHEMA_HISTORY \
--partition-count 1 \
--cleanup-policy Delete \
--retention-time-in-hours 168 \
--output none
```
This seems like terrible advice for any production environment.
Shouldn't it be cleanup-policy compact and infinite retention?
however, when I set this I cannot get debezium to write to the topic/hub without CORRUPT_MESSAGE errors. Works fine when --cleanup-policy Delete
Contributor guide
Research direction
Start with the Azure CLI Event Hubs creation command in the issue and compare the Delete configuration with Compact and the retention setting. Reproduce the Debezium write using each configuration and investigate the reported CORRUPT_MESSAGE errors; the work is done when the sample configuration or its documented limitation is clear and verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, kafka
- Domain
- cloud, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100