dragonflydb / dragonflydb/dragonfly

Software error on replication

Open
#3,170 36 comments 1 reaction 1 assignee View on GitHub

Nobody has claimed this yet.

bug important
Dominant language
C++
Stars
31.6k
Forks
1.3k
Avg merge
1d 10h
Merged PRs (30d)
137

Description

I have a 3 nodes running dragonfly 1 master and 2 replicas.
The access to dragonfly instances is over haproxy with 2 backend: 1 for write (master) and 1 for read (master/replica)

The kernel version on 1 server is 5.15.0-107-generic on all servers and 5.15.0-112-generic on others

The hardware is:
Master: 96 cores and 376 Gb ram
Replica 1: 48 cores and 256 Gb ram
Replica 1: 48 cores and 256 Gb ram

DragronFly version v1.19.0

Since the migration to 1.19.0 the nodes are very unstable, I frequently find this error in the logs:
`
I20240612 18:18:28.297003 3847833 dflycmd.cc:577] Replication error: Software caused connection abort
I20240612 18:18:28.964834 3847826 dflycmd.cc:640] Disconnecting from replica 172.16.0.25:6392
I20240612 18:18:58.025323 3847831 server_family.cc:2435] Replicating 172.16.0.25:6392
I20240612 18:18:58.285149 3847831 replica.cc:533] Started full sync with 172.16.0.25:6392
I20240612 18:19:03.880859 3847831 replica.cc:553] full sync finished in 5.66 s
I20240612 18:19:03.881709 3847831 replica.cc:634] Transitioned into stable sync
I20240612 18:19:56.525679 3847826 dflycmd.cc:640] Disconnecting from replica 172.16.0.23:6392
`
The service is balanced by the inconvenience that it is.
There are no more traces in other logs beyond the sentinel log.
thank you

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.