StackExchange / StackExchange/StackExchange.Redis
Abnormal master-slave switching in sentinelmode
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 6.2k
- Forks
- 1.6k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 43
Description
Thank you very much for your contribution. I encountered some difficulties when using it and would like to ask for advice.
My redis configuration is 1 master A, 1 slave B, 3 sentinels.
In sentinel mode, I found that after node A went offline, node B was upgraded to the master node.
There was no problem in this process.
But when the old master node A comes online again, stackExchange.redis will briefly mark both nodes as primary, and use the old master node A.
But A is converting to a slave node at this time and synchronizing new data. This will lead to the error of reading and writing A at this time.
The program will mark A as a replica later, and the error will be corrected.
Regarding this problem, how should I solve it? Is there any relevant configuration that can solve this problem?
MyTestCode
using StackExchange.Redis;
using static StackExchange.Redis.Role;
namespace ConsoleApp3
{
internal class Program
{
static async Task Main(string[] args)
{
Console.WriteLine("Hello, World!");
ConnectionMultiplexer masterConnection = ConnectionMultiplexer.Connect("172.16.184.197:26379,172.16.184.198:26379,172.16.0.29:26379,serviceName=mymaster");
while (true)
{
try
{
IDatabase database = masterConnection.GetDatabase();
await database.StringIncrementAsync("SomeKey", 1);
System.Console.WriteLine(masterConnection.GetStatus());
System.Console.WriteLine(DateTime.Now.ToString("yyyyMMddHHmmss::") + await database.StringGetAsync("SomeKey"));
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
await Task.Delay(1000);
}
}
}
}
Start
stop A and switch to B
start A and the data is dirty
recover normal
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Use the C# sample's ConnectionMultiplexer.Connect(...) with serviceName=mymaster and the three Sentinel endpoints to reproduce the failover. Start by tracing how Sentinel role changes are observed after A returns and how GetDatabase() routes StringIncrementAsync. Done means operations are not sent to A while it is rejoining as a replica.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, redis
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100