[Bug] [Config] Inconsistent defaults for storage flood stage thresholds between FE and BE
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
v3.1.4
### What's Wrong?
[Config] Inconsistent defaults for storage flood stage thresholds between FE and BE
Description
Apache Doris defines storage_flood_stage_usage_percent and storage_flood_stage_left_capacity_bytes in both FE and BE configuration. The two components use the same parameter names, but the default value of storage_flood_stage_usage_percent is inconsistent.
According to the official documentation:
Parameter | FE default | BE default
-- | -- | --
storage_flood_stage_usage_percent | 95 | 90
storage_flood_stage_left_capacity_bytes | 1 GiB | 1 GiB
References:
Problem
### Conflicting Configuration Defaults Between FE and BE
The same configuration parameters have **conflicting defaults in FE and BE**, which can mislead users when configuring a Doris cluster.
For example, at **92% disk usage**:
* **FE:** Stays normal (default threshold is **95%**).
* **BE:** Triggers flood-stage protection and returns errors (default threshold is **90%**).
#### Impact
This mismatch causes **unexpected query failures**. It makes disk protection behavior unpredictable and complicates both cluster configuration and troubleshooting.
### What You Expected?
The default values and descriptions of the corresponding FE and BE parameters should be consistent, or the documentation should clearly explain why they intentionally differ.
### How to Reproduce?
- set config::storage_flood_stage_usage_percent 92% as hard-coded.
- set storage_flood_stage_left_capacity_bytes to a high value enough for satisy this condition in data_dir.cpp:
```
bool DataDir::reach_capacity_limit(int64_t incoming_data_size) {
double used_pct = get_usage(incoming_data_size);
int64_t left_bytes = _available_bytes - incoming_data_size;
if (used_pct >= config::storage_flood_stage_usage_percent / 100.0 &&
left_bytes <= config::storage_flood_stage_left_capacity_bytes) {
LOG(WARNING) << "reach capacity limit. used pct: " << used_pct
<< ", left bytes: " << left_bytes << ", path: " << _path;
return true;
}
return false;
}
```
- then the backend will return an error. It is still treated as a query execution candidate, though, since exceedLimit continues to return false:
```
public boolean exceedLimit(boolean floodStage) {
if (LOG.isDebugEnabled()) {
LOG.debug("flood stage: {}, diskAvailableCapacityB: {}, totalCapacityB: {}",
floodStage, diskAvailableCapacityB, totalCapacityB);
}
if (floodStage) {
return diskAvailableCapacityB < Config.storage_flood_stage_left_capacity_bytes
&& this.getUsedPct() > (Config.storage_flood_stage_usage_percent / 100.0);
} else {
return diskAvailableCapacityB < Config.storage_min_left_capacity_bytes
|| this.getUsedPct() > (Config.storage_high_watermark_usage_percent / 100.0);
}
}
```
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Compare the defaults in FE Config.java and the BE implementation in data_dir.cpp, then read the linked FE and BE configuration documentation. Confirm whether the 95% and 90% values are intentional and trace the documented behavior around storage_flood_stage_usage_percent and storage_flood_stage_left_capacity_bytes. Done means the FE and BE defaults and descriptions are consistent, or their intentional difference is clearly documented, with tests or verification covering the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, java
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100