apache / apache/ignite

Consistent hashing algorithm leads to uneven distribution of partitions.

Open
#12,296 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
5.1k
Forks
1.9k
Avg merge
3d 2h
Merged PRs (30d)
46

Description

Hello, we are using Ignite 2.15 in our project. In practice, we found that the partition assignment method RendezvousAffinityFunction, which is based on consistent hashing, has an issue with uneven distribution. For example, when assigning partitions to 120 nodes, the number of partitions assigned to each node varies significantly. Is there any way to solve this problem?

Alternatively, if our project does not need to consider backups or data migration caused by nodes joining or leaving the cluster, can we simply use a straightforward hash function instead of consistent hashing for partition assignment? If so, what potential issues might arise from this approach?

Image

[RendezvousAffinityFunctionTest.java](https://github.com/user-attachments/files/21968975/RendezvousAffinityFunctionTest.java)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing RendezvousAffinityFunction and the attached RendezvousAffinityFunctionTest.java, focusing on how partitions are assigned across 120 nodes. Compare the observed distribution with the intended assignment guarantees; the issue is complete only once the expected behavior and an agreed remedy are defined and verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.