influxdata / influxdata/influxdb

RAM is not cleaned after task finish

Open
#20,756 2 comments 0 reactions 0 assignees View on GitHub
area/2.x area/memory area/queries
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

1. Create task that needs a lot of resources (ex. copying few milion points to new bucket)
2. Run task

Here is the task that I run:
```
option task = {
name: "Przeniesienie 10s 2020",
every: 999h,
}

data = from(bucket: "Stacja_pogodowa/autogen")
|> range(start: 2020-01-01T00:00:00Z, stop: 2021-01-01T00:00:00Z)
|> filter(fn: (r) =>
(r._field == "Wind_speed" or r._field == "Wind_speed_max"))
|> map(fn: (r) => ({
r with _field: if r._field == "Wind_speed" then "instantaneous_wind_speed"
else if r._field == "Wind_speed_max" then "wind_gust" else r._field,
_measurement: "meteo"}))
|> truncateTimeColumn(unit: 1s)

data
|> to(bucket: "meteo")
```

__Expected behavior:__
After the task is finished, RAM should be cleaned

__Actual behavior:__
RAM is not getting cleaned or is it getting cleaned very slowly. Docker is using 10GB of RAM a few minutes after task succeded:
![image](https://user-images.githubusercontent.com/19311511/107860591-52bccf80-6e40-11eb-9cfd-2cb79abeab88.png)

20 minutes after task finished:
![image](https://user-images.githubusercontent.com/19311511/107860819-fc509080-6e41-11eb-840b-b6d4c4604fad.png)

Before running the task Docker was using about 1GB of RAM

The same behaviour occured on arm64 Ubuntu (Raspberry Pi 3B)

__Environment info:__

* System info: Windows 10 + Docker 3.1.0
* InfluxDB version: 2.0.4
* Other relevant environment details: i7-9700K processor, 16GB RAM DDR4, NVME drive

__Performance:__
[profiles.tar.gz](https://github.com/influxdata/influxdb/files/5976471/profiles.tar.gz)
[vars.txt](https://github.com/influxdata/influxdb/files/5976472/vars.txt)
Unfortunately, I cannot run iostat on Docker

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the task shown in the issue on InfluxDB 2.0.4 and review the supplied profiles.tar.gz and vars.txt for the post-task memory behavior. Compare Docker memory before execution, immediately after success, and 20 minutes later. Done means determining why memory remains elevated or is released slowly and verifying the expected cleanup after the task finishes.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.