Azure / Azure/azure-documentdb-changefeedprocessor-dotnet
Stream failure doesn't appear to restart the lease
- Dominant language
- C#
- Stars
- 51
- Forks
- 24
- PR merge metrics
- No merged PRs in 30d
Description
We are having an issue where stream processing failures are throwing an exception, restarting the service, appear to reacquire the lease, but the stream basically goes into a mode where it appears alive but nothing is ever received. It *may* be that if the restart occurs inside a certain time window, it thinks the lease is normal, but it ends up in this weird "zombie" mode where nothing is received. Our solution for now has been to restart the service again, which seems to work, suggesting it may be a timing issue.
Any suggestions for how to deal with this appropriately?
I have listed the logs below:
21:04:42 [Information] Partition "0": renewed lease with result True
21:04:50 [Information] Host '"host-4b8033ae-ae23-409b-9e97-2d2c4963eb77"' 1 partitions, 1 hosts, 0 available leases, target = 1, min = 0, max = 0, mine = 1, will try to take 0 lease(s) for myself'.
-- exception --
21:04:54 [Information] Stopping processor...
21:04:54 [Information] Partition load balancer task stopped.
-- service restarted -- (*notice there is no acquire lease info logs though!*)
21:04:59 [Information] Partition "0": renewed lease with result True
21:05:16 [Information] Partition "0": renewed lease with result True
21:05:33 [Information] Partition "0": renewed lease with result True
21:05:50 [Information] Partition "0": renewed lease with result True
21:06:07 [Information] Partition "0": renewed lease with result True
21:06:24 [Information] Partition "0": renewed lease with result True
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.