apache / apache/iotdb

[Bug] Limiting memory does not degrade query latency

未关闭
#17,070 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Java
星标
6.4k
派生
1.2k
平均合并
1 天 23 小时
30 天内合并 PR
115

描述

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/iotdb/issues) and found nothing similar.

### Version

1.1.3 and 1.3.4 (tested with standalone Docker image)

### Describe the bug and provide the minimal reproduce step

While testing IoTDB under limited memory configurations, using docker for that effect, it was found that the database's performance won't drop until the threshold of 4GB of memory is hit, despite it always using the maximum amount of memory allocated for the container.

It was expected that progressively constraining the database's available memory would lead to gradual performance degradation, but instead IoTDB maintains virtually identical query latency across the tested memory limits, despite always consuming the maximum amount of memory available to the container.

Minimal reprodution steps:

1. Launch IoTDB in standalone mode with constrained memory using Docker Compose
2. Preload data before executing queries
3. Execue a query workload, using the same for every memory configuration

The queries follow the following templates:
```sql
-- Aggregation
select agg_func(field) from path where time >= start and time <= end

-- Downsampling
select field from path group by ([start, end), step)

-- Outlier-filter
select field from path where time >= start and time <= end and field [>,>=,<,<=] threshold
```

Image

Image

### What did you expect to see?

Either performance degradation as memory limits get increasingly smaller, or the container not using all the memory available to it, if it is able to maintain performance with less memory usage.

### What did you see instead?

Performance didn't degrade until the 4GB memory limit, remaining similar regardless of the limit used, despite the database always using the maximum memory allocated to it.

### Anything else?

The tests were run using [MulletBench](https://github.com/pedropereira98/MulletBench/tree/main), as well as plot generation.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

贡献指南

打开贡献指南

调研方向

首先,使用 issue 中描述的 Docker Compose 内存配置复现 MulletBench 报告的工作负载,并比较查询延迟与容器内存使用量。追踪导致不同限制下延迟保持不变的数据库行为;完成的标准是确定原因,并针对聚合、降采样和异常值过滤工作负载验证修复方案。

由索引模型根据 Issue 内容生成。

评估

技术栈
docker, java, sql
领域
databases, performance
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
需要澄清
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。