OOM with Influx 2.0.7 running inside a docker container with memory constrains

I'm facing an issue with influx 2.0.7 running in a docker container with memory constrains, I'm aware of memory tradeoffs for databases, and I'm ok with it being slower due to limited resources. I tried to limit all memory/buffer parameters that I could to keep it under control, but influx keeps dying from OOM while importing big historical datasets (1.5mi data points) into 5 measurements with about 5 tags.

Bellow my deployment code (ansible)

```yaml
- name: "Monitoring | Influxdb"
docker_container:
name: influx
image: influxdb:2.0.7
restart_policy: unless-stopped
memory: 4g
#memory_swap: 5
#memory_swappiness: 5
env:
INFLUXDB_UDP_ENABLED: "true"
INFLUXDB_REPORTING_DISABLED: "true"
INFLUXDB_DATA_CACHE_MAX_MEMORY_SIZE: "2G"
INFLUXDB_DATA_INDEX_VERSION: "tsi1"
INFLUXDB_CONFIG_PATH: "/etc/influxdb2/influxdb.config"
command:
- "--storage-max-concurrent-compactions=1"
- "--storage-series-file-max-concurrent-snapshot-compactions=1"
- "--storage-compact-full-write-cold-duration=2h"
- "--storage-cache-snapshot-write-cold-duration=5m"
- "--query-max-memory-bytes=107374182"
- "--http-read-timeout=0"
- "--http-write-timeout=0"
- "--bolt-path=/data/bolt"
- "--engine-path=/data/engine"
#- "--storage-series-id-set-cache-size=1024"
- "--storage-retention-check-interval=48h"
- "--query-queue-size=102400"
- "--query-concurrency=16"
- "--reporting-disabled=true"
ports:
- "8086:8086"
- "8089:8089/udp"
volumes:
- "/data/backups/:/backups"
- "influx-data:/data"
- "/config/influx-configs:/etc/influxdb2/influx-configs"

```

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

1. Start influx with above command
2. Try to load some historical datasets
3. After a while

```
[605511.957732] influxd invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=0, oom_score_adj=0
[605511.957735] influxd cpuset=ebfab5d905bd35fddc9a6931e6b784d213f51c0a180292e17c1b4594ed15dffc mems_allowed=0
[605511.957746] CPU: 2 PID: 17527 Comm: influxd Not tainted 4.19.0-16-amd64 #1 Debian 4.19.181-1
[605511.957748] Hardware name: Dell Inc. OptiPlex 3050/0JP3NX, BIOS 1.15.1 12/22/2020
[605511.957749] Call Trace:
[605511.957763] dump_stack+0x66/0x81
[605511.957768] dump_header+0x6b/0x283
[605511.957774] oom_kill_process.cold.30+0xb/0x1cf
[605511.957782] out_of_memory+0x1a5/0x450
[605511.957788] mem_cgroup_out_of_memory+0xbe/0xd0
[605511.957794] try_charge+0x63a/0x780
[605511.957801] mem_cgroup_try_charge+0x86/0x190
[605511.957807] ? pagecache_get_page+0x30/0x2c0
[605511.957812] mem_cgroup_try_charge_delay+0x1c/0x40
[605511.957817] do_swap_page+0x224/0x8d0
[605511.957822] __handle_mm_fault+0x87c/0x11f0
[605511.957827] ? __switch_to_asm+0x41/0x70
[605511.957833] handle_mm_fault+0xd6/0x200
[605511.957838] __do_page_fault+0x249/0x4f0
[605511.957844] ? page_fault+0x8/0x30
[605511.957849] page_fault+0x1e/0x30
[605511.957854] RIP: 0033:0x4272e6
[605511.957858] Code: ff c6 0f b6 3b 49 89 cb 89 f1 41 89 fc d3 ef 49 83 fb 08 74 0a 0f ba e7 04 0f 83 c3 00 00 00 41 0f a3 cc 90 73 af 4b 8d 3c 0b <48> 8b 3f 48 85 ff 74 a3 49 89 fc 4c 29 cf 48 39 d7 72 98 48 89 5c
[605511.957861] RSP: 002b:00007f65058bf850 EFLAGS: 00010247
[605511.957864] RAX: 0000000000203016 RBX: 00007f64fa460dc8 RCX: 0000000000000001
[605511.957867] RDX: 0000000000001300 RSI: 0000000000000001 RDI: 000000c05941b908
[605511.957869] RBP: 00007f65058bf8d0 R08: 00007f64fa5bffff R09: 000000c05941b900
[605511.957871] R10: 000000c000062698 R11: 0000000000000008 R12: 00000000000000da
[605511.957872] R13: 00000000058448e0 R14: 0000000000000000 R15: 0000000000000000
[605511.957876] Task in /docker/ebfab5d905bd35fddc9a6931e6b784d213f51c0a180292e17c1b4594ed15dffc killed as a result of limit of /docker/ebfab5d905bd35fddc9a6931e6b784d213f51c0a180292e17c1b4594ed15dffc
[605511.957885] memory: usage 3983680kB, limit 4194304kB, failcnt 46943
[605511.957888] memory+swap: usage 8388608kB, limit 8388608kB, failcnt 2457
[605511.957890] kmem: usage 37268kB, limit 9007199254740988kB, failcnt 0
[605511.957891] Memory cgroup stats for /docker/ebfab5d905bd35fddc9a6931e6b784d213f51c0a180292e17c1b4594ed15dffc: cache:28KB rss:3946172KB rss_huge:1671168KB shmem:0KB mapped_file:0KB dirty:0KB writeback:0KB swap:4405216KB inactive_anon:657844KB active_anon:3288512KB inactive_file:52KB active_file:4KB unevictable:0KB
[605511.957906] Tasks state (memory values in pages):
[605511.957907] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[605511.958069] [ 16803] 1000 16803 2572919 997119 18964480 1103566 0 influxd
[605511.958075] Memory cgroup out of memory: Kill process 16803 (influxd) score 1003 or sacrifice child
[605511.958169] Killed process 16803 (influxd) total-vm:10291676kB, anon-rss:3935536kB, file-rss:52940kB, shmem-rss:0kB
[605513.118538] oom_reaper: reaped process 16803 (influxd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
```

__Expected behavior:__
Guidance on how to run influxdb with limited memory constrains without suffering such problems.

__Actual behavior:__
Influx dies and get stuck into a restart loop due to OOM.

__Environment info:__

* System info: `Linux 4.19.0-16-amd64 x86_64`
* InfluxDB version: `InfluxDB 2.0.7 (git: 2a45f0c037) build_date: 2021-06-04T19:17:40Z`
* Other relevant environment details: Running on docker, in a Debian linux on a bare metal machine.

__Config:__
Configuration is set on command line arguments on the above snippet

__Logs:__
Logs are quite big and with a lot of very similar entries as bellow

```
ts=2021-06-30T06:13:52.067412Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2205 duration=38.368ms
ts=2021-06-30T06:13:52.070047Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2150/_00001.wal size=7009272
ts=2021-06-30T06:13:52.100446Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:13:52.103596Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2207/000000008-000000001.tsm id=1 duration=0.427ms
ts=2021-06-30T06:13:52.105162Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:13:52.105155Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2207/000000006-000000002.tsm id=0 duration=1.140ms
ts=2021-06-30T06:13:52.106545Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2207 duration=38.987ms
ts=2021-06-30T06:13:52.129169Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2206/000000006-000000002.tsm id=0 duration=15.406ms
ts=2021-06-30T06:13:52.129476Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:13:52.129518Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2206 duration=78.383ms
ts=2021-06-30T06:13:52.157937Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2208/000000006-000000002.tsm id=0 duration=11.988ms
ts=2021-06-30T06:13:52.158635Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2208 duration=51.819ms
ts=2021-06-30T06:13:52.215262Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:13:52.226000Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2151/_00001.wal size=7610704
ts=2021-06-30T06:13:52.253481Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:13:52.262175Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2152/_00001.wal size=6853353
ts=2021-06-30T06:14:05.797081Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2149 duration=13868.619ms
ts=2021-06-30T06:14:05.870651Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:05.882055Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2209/000000007-000000001.tsm id=1 duration=0.894ms
ts=2021-06-30T06:14:05.886611Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2209/000000005-000000002.tsm id=0 duration=3.016ms
ts=2021-06-30T06:14:05.888572Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2209 duration=89.604ms
ts=2021-06-30T06:14:06.038251Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.047253Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2153/_00001.wal size=6063149
ts=2021-06-30T06:14:06.194766Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2150 duration=14194.929ms
ts=2021-06-30T06:14:06.252163Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.255755Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/221/_00001.wal size=58935
ts=2021-06-30T06:14:06.334527Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/221 duration=139.432ms
ts=2021-06-30T06:14:06.445857Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2152 duration=14286.931ms
ts=2021-06-30T06:14:06.495199Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.499168Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2151 duration=14369.603ms
ts=2021-06-30T06:14:06.499803Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2154/_00001.wal size=6955768
ts=2021-06-30T06:14:06.532590Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.556118Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2210/000000006-000000002.tsm id=0 duration=2.915ms
ts=2021-06-30T06:14:06.556807Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2210 duration=109.049ms
ts=2021-06-30T06:14:06.619907Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.629603Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2211/000000007-000000001.tsm id=1 duration=0.832ms
ts=2021-06-30T06:14:06.632960Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2211/000000005-000000002.tsm id=0 duration=3.081ms
ts=2021-06-30T06:14:06.633474Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2211 duration=76.374ms
ts=2021-06-30T06:14:06.672583Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.678932Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2155/_00001.wal size=7561101
ts=2021-06-30T06:14:06.787156Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.801331Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2156/000000002-000000002.tsm id=0 duration=11.179ms
ts=2021-06-30T06:14:06.801960Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2156/_00005.wal size=9032745
#
```

__Performance:__
Due to the fact that influxdb get stuck into a restart loop and don't start the http interface makes hard to run the command to get profiler information. I'm awaiting for further instructions on how to do it.

```sh
# Commands should be run when the bug is actively happening.
# Note: This command will run for ~30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=30s"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz` and `iostat.txt` output files.
```

Start by reproducing the historical import with the posted Docker and Ansible configuration, then inspect the cgroup OOM output and InfluxDB 2.0.7 storage logs. Compare the configured memory, query, and compaction limits with the observed usage. Done means the failure is explained and a supported way to complete the import within the container limit is documented or validated.

Written by the indexing model from the issue text.