StackExchange / StackExchange/StackExchange.Redis
StringIncrement() returning the same value on two different machines
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 6.2k
- Forks
- 1.6k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 43
Description
We are using StackExchange.Redis (v2.0.601) to connect to Azure Redis. We are using StringIncrement() to increment the value of a key while inserting a record into another storage system, and then stop inserting when the Redis value exceeds a certain number. This is running on multiple machines with each service multi-threaded. This is a high load scenario where we are hitting StringIncrement() with hundreds or even thousands of requests per second, spread across multiple machines.
What I am seeing is that we are actually inserting more records than we expect. When I log the values being returned by StringIncrement(), ** I see that we are getting the same value returned on different machines**. Here is the code we are executing:
returnedValue = AzureCache(clusterIndex).StringIncrement(cacheKey, quantity)
The list below is shows a count of the value returned and the count of them.
|returned-value-after-increment|count-of-this-value-across-all-machines|
So, the value 208 was returned to 4 different threads. The value 3 was returned to 3 different threads.
| 208 | 4 |
| 207 | 4 |
| 211 | 4 |
| 209 | 4 |
| 206 | 4 |
| 210 | 4 |
| 205 | 4 |
| 213 | 4 |
| 212 | 4 |
| 3 | 3 |
| 4 | 3 |
| 215 | 3 |
| 2 | 3
I thought that StringIncrement() is atomic and could never return the same value to two different threads/machines. Can anyone explain what is going on and how to fix this issue? Am I misunderstanding how this should work?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file or test is named. Start from the StringIncrement(cacheKey, quantity) call against Azure Redis and reproduce the high-load, multi-machine scenario described with StackExchange.Redis v2.0.601; done means explaining the repeated returned values and identifying a verified fix or limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp, redis
- Domain
- backend, databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100