DT.AzureStorage: wrong blob may win when split-brain instances race
- Dominant language
- C#
- Stars
- 1.7k
- Forks
- 335
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 6
Description
In split-brain situations, two hosts may try to write the same instance history simultaneously.
To protect against state corruption, the `AzureTableTrackingStore` uses an e-Tag, stored in a sentinel row, which ensures that only one of the racing accesses can actually update the table. The losing access will then abandon the orchestration work item, thus resolving the split-brain situation (only the winner remains active).
Unfortunately, this mechanism is not sufficient to also protect against *races on blobs* that can happen when table properties are too large and are stored in a blob. In that case, the losing access (in terms of e-tag) may still be able to overwrite the blob of the winning access, leading to incorrect results as reported in https://github.com/Azure/azure-functions-durable-extension/issues/2437.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.