aws / aws/aws-advanced-jdbc-wrapper
Load-aware host selectors rank on stale cpuPercent/lagMs after a topology refresh
- Dominant language
- Java
- Stars
- 358
- Forks
- 90
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 32
Description
### Describe the bug
PluginService.getHosts() continuously reports the weight, cpuPercent, and lagMs captured during the last structural topology change. When ClusterTopologyMonitorImpl publishes a refresh where only measured metrics change, freshly fetched HostSpec instances are discarded while previously published ones are retained.
As a result, all load-aware host selectors (lowestLoad, lowestLoadByCpu, lowestLoadByLag, highestLoad, highestLoadByCpu, highestLoadByLag, weightedRandom, and deprecated highestWeight) evaluate stale values indefinitely on clusters with stable membership.
The issue stems from a structural mismatch between two comparison methods:
PluginServiceImpl.compare(HostSpec, HostSpec) (PluginServiceImpl.java:421) checks hostname, port, role, and availability, but ignores weight, cpuPercent, and lagMs.
setNodeList (PluginServiceImpl.java:597) updates allHosts only inside the block that triggers notifyNodeListChanged (PluginServiceImpl.java:632):
Java
if (!changes.isEmpty()) {
this.allHosts = newHosts != null ? newHosts : new ArrayList<>();
this.pluginManager.notifyNodeListChanged(changes);
}
When a refresh carries only updated measurements, changes evaluates to empty, causing the freshly fetched HostSpec list to be dropped on the floor.
### Expected Behavior
PluginService.getHosts() should immediately reflect current weight, cpuPercent, and lagMs metrics after the topology monitor publishes a new snapshot.
Load-aware selectors (such as lowestLoadByLag) should react promptly to reader performance changes (e.g., stopping selection of a reader whose replication lag increases once observed by the monitor).
HostSpec data within active PluginService instances should be kept continuously fresh whenever ClusterTopologyMonitorImpl updates its cached topology.
### What plugins are used? What other connection properties were set?
autoReadWriteSplitting,
### Current Behavior
No exception or log entry is produced; the wrapper silently routes connections using outdated metric data. On a stable Aurora cluster with readerHostSelectorStrategy=lowestLoadByLag:
ClusterTopologyMonitorImpl.updateTopologyCache updates the Topology entry in the storage service every clusterTopologyRefreshRateMs.
getHosts() continues returning lag values from the last structural membership update.
LowestLoadHostSelector.calculateLoad scores on stale lag, leaving reader selection unable to react to lagging replicas.
### Reproduction Steps
Connect to an Aurora cluster with at least two readers and fixed membership.
Set readerHostSelectorStrategy=lowestLoadByLag.
Apply write load to the primary writer to increase replication lag on the active reader.
Compare pluginService.getHosts() output against the storage service's Topology entry: the cached topology updates while getHosts() metrics remain frozen.
### Possible Solution
_No response_
### Additional Information/Context
_No response_
### The AWS Advanced JDBC Wrapper version used
v4.4.0
### JDK version used
openjdk version "21.0.12" 2026-07-21 LTS (Corretto-21.0.12.8.1)
### Operating System and version
macOS 26.5.2 (build 25F84)
Contributor guide
Research direction
Review PluginServiceImpl.java at compare(...) and setNodeList(...), then trace ClusterTopologyMonitorImpl.updateTopologyCache and PluginService.getHosts(). Reproduce with the described fixed-membership Aurora setup and compare getHosts() with the storage Topology entry. Done means fresh weight, cpuPercent, and lagMs reach getHosts() and load-aware selectors respond to changed lag.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- backend, databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100