AMM: select candidates based on memory percentage, not absolute usage?
Open
memory
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
The `ActiveMemoryManagerExtension` currently selects candidates to receive or drop replicates based on the workers with the lowest/highest optimistic memory, in bytes.
In heterogeneous clusters, doesn't this bias AMM towards small workers being under more memory pressure than large workers?
For example:
* worker A is using 5GiB, and has 10GiB capacity
* worker B is using 8GiB, and has 100GiB capacity
`find_recipient` would pick worker A to replicate keys onto, even though it's under more memory pressure.
`find_dropper` would pick worker B to drop from, even though it's under much less memory pressure.
cc @crusaderky
Contributor guide
Assessment
This issue has not been assessed yet.