Allow for heterogeneous hardware clusters
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 487
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 13
Description
Currently, when settings are set in the accumulo configs for tserver based properties they apply across the cluster. We are moving toward replacing clusters through rolling upgrades, and at any time may have different machine types running tservers at a given time. Right now that causes us to leave hardware idle because we cannot change settings on only a subset of tservers. Settings which we might want to change depending on the hardware a tserver is running on include, but are not limited to: tserver.cache.*.size, tserver.scan.files.open.max, tserver.compaction.major.concurrent.max etc.
The goal of this issue is to allow tservers running on nodes of a different hardware type to use class based overrides for tserver properties.
- Add ability to add a tserver.class. property where the value is a regex that when matched will put the tserver in this class. EG: tserver.class.beefymachines=bigmachine.*
- Add ability to add .tserver.[property] to override only for this class EG beefymachines.tserver.compaction.major.concurrent.max=10
- Add ability to retrieve class of tserver from config shell command. EG: config --tserver bigMachine123 would print class: beefymachines and then the properties as overridden by that class
- Add ability to retrieve config for a class of machine. EG config --tserver-class beefymachines
Contributor guide
Research direction
Start by tracing how Accumulo tserver properties are loaded and how the config shell command reports them. The issue specifies class regexes, class-specific tserver overrides, and both --tserver and --tserver-class lookups; done means these behaviors work during mixed-hardware rolling upgrades.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- cli, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100