apache / apache/iotdb

[Bug] Limiting memory does not degrade query latency

Abierto
#17,070 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Java
Estrellas
6.4k
Forks
1.2k
Merge medio
1 d 23 h
PR fusionados (30 d)
115

Descripción

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/iotdb/issues) and found nothing similar.

### Version

1.1.3 and 1.3.4 (tested with standalone Docker image)

### Describe the bug and provide the minimal reproduce step

While testing IoTDB under limited memory configurations, using docker for that effect, it was found that the database's performance won't drop until the threshold of 4GB of memory is hit, despite it always using the maximum amount of memory allocated for the container.

It was expected that progressively constraining the database's available memory would lead to gradual performance degradation, but instead IoTDB maintains virtually identical query latency across the tested memory limits, despite always consuming the maximum amount of memory available to the container.

Minimal reprodution steps:

1. Launch IoTDB in standalone mode with constrained memory using Docker Compose
2. Preload data before executing queries
3. Execue a query workload, using the same for every memory configuration

The queries follow the following templates:
```sql
-- Aggregation
select agg_func(field) from path where time >= start and time <= end

-- Downsampling
select field from path group by ([start, end), step)

-- Outlier-filter
select field from path where time >= start and time <= end and field [>,>=,<,<=] threshold
```

Image

Image

### What did you expect to see?

Either performance degradation as memory limits get increasingly smaller, or the container not using all the memory available to it, if it is able to maintain performance with less memory usage.

### What did you see instead?

Performance didn't degrade until the 4GB memory limit, remaining similar regardless of the limit used, despite the database always using the maximum memory allocated to it.

### Anything else?

The tests were run using [MulletBench](https://github.com/pedropereira98/MulletBench/tree/main), as well as plot generation.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Comienza reproduciendo la carga de trabajo reportada de MulletBench mediante las configuraciones de memoria de Docker Compose descritas en la issue, y compara la latencia de las consultas con el uso de memoria del contenedor. Rastrea el comportamiento de la base de datos responsable de que la latencia no cambie entre los límites; se considera terminado cuando se haya identificado la causa y validado una corrección frente a las cargas de trabajo de agregación, downsampling y filtrado de valores atípicos.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
docker, java, sql
Área
databases, performance
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Necesita aclaración
Aptitud para principiantes
35/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.