StackExchange / StackExchange/StackExchange.Redis
[BUG] Batches are executed out of order when using DNS connections
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 6.2k
- Forks
- 1.6k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 43
Description
Backgroud
Our user told us that when they use batch on our cloud (AlibabaCloud), their commands came out of order. code is below
namespace RedisTest;
using StackExchange.Redis;
using System;
using System.Collections.Generic;
using System.Linq;
public class RedisClient
{
class Program
{
static void Main(string[] args)
{
ConfigurationOptions configurationOptions = ConfigurationOptions.Parse("r-bpxxxxpd.redis.rds.aliyuncs.com:6379,password=xxx,connectTimeout=2000");
ConnectionMultiplexer redisConn = ConnectionMultiplexer.Connect(configurationOptions);
var db = redisConn.GetDatabase();
var batch = db.CreateBatch();
batch.KeyDeleteAsync("testhash"); // UNLINK
var keyval = new List<HashEntry>(10);
for (int i = 0; i < 10; i++) {
keyval.Add(new HashEntry(i, i));
}
batch.HashSetAsync("testhash", keyval.ToArray()); // HMSET
batch.KeyExpireAsync("testhash", TimeSpan.FromSeconds(100)); // EXPIRE
batch.Execute();
}
}
}
Ideally, they would get the following sequence:
- UNLINK
- HMSET
- EXPIRE
But sometimes, the following sequence is produced:
- UNLINK
- EXPIRE
- HMSET
Going a step further, we found that UNLINK and EXPIRE are sent over one TCP connection, but HMSET uses another TCP connection.
Why are there two links? We found that when connecting through a domain name, there will be two links in ConnectionMultiplexer#ServerSnapshot (one is the domain name, and the other is the IP resolved by the domain name (obtained through cluster nodes)). Therefore, two links will be returned randomly in AnyServer, causing commands to be assigned to different connections.
How to reproduction
- Find a server with DNS (I can provide Alibaba Cloud test environment for free)
- Run the above command
How to fix
During the initialization process, Lettuce will generate the result of the domain name as a URI, but will change the domain name to an alias. Please refer to https://github.com/redis/lettuce/commit/16f9e7525068a3887f5cc746c6b976720835600a
Version
- StackExchange.Redis: latest
- .Net: I reproduced this issue under 6.0 and 8.0, but it has nothing to do with the .net version.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided C# reproduction using a DNS-based AlibabaCloud Redis endpoint, then inspect ConnectionMultiplexer.ServerSnapshot and AnyServer, where the issue reports duplicate hostname and resolved-IP entries. Compare the selected connections and captured command order against the expected UNLINK, HMSET, and EXPIRE sequence; done means the DNS case no longer sends one batch across separate connections or reorders those commands.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, redis
- Domain
- databases, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100