apache / apache/accumulo

Allow for heterogeneous hardware clusters

Open
#1,265 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.2k
Forks
487
Avg merge
4d 5h
Merged PRs (30d)
13

Description

Currently, when settings are set in the accumulo configs for tserver based properties they apply across the cluster. We are moving toward replacing clusters through rolling upgrades, and at any time may have different machine types running tservers at a given time. Right now that causes us to leave hardware idle because we cannot change settings on only a subset of tservers. Settings which we might want to change depending on the hardware a tserver is running on include, but are not limited to: tserver.cache.*.size, tserver.scan.files.open.max, tserver.compaction.major.concurrent.max etc.

The goal of this issue is to allow tservers running on nodes of a different hardware type to use class based overrides for tserver properties.

- Add ability to add a tserver.class. property where the value is a regex that when matched will put the tserver in this class. EG: tserver.class.beefymachines=bigmachine.*
- Add ability to add .tserver.[property] to override only for this class EG beefymachines.tserver.compaction.major.concurrent.max=10
- Add ability to retrieve class of tserver from config shell command. EG: config --tserver bigMachine123 would print class: beefymachines and then the properties as overridden by that class
- Add ability to retrieve config for a class of machine. EG config --tserver-class beefymachines

Contributor guide

Open the contributing guide

Research direction

Start by tracing how Accumulo tserver properties are loaded and how the config shell command reports them. The issue specifies class regexes, class-specific tserver overrides, and both --tserver and --tserver-class lookups; done means these behaviors work during mixed-hardware rolling upgrades.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
cli, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.