influxd keeps high memory usage

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

Use influx-stress to generate testing data. I have modified the source code to generate last n months data.
Data format example:

name,tags,time,n,tag_src_MAC,tag_src_ip
test,,1610850496189209576,31,32:ED:56:27:23:d4,10.64.35.212
test,,1610850498998543985,117,32:ED:56:27:0b:3d,10.64.11.61

https://github.com/influxdata/influx-stress

1. Use influx-stress to generate last 6 months data. Total amount is 10 millions data.
2. Wait until TSM compaction end, and restart influxd. (Let influxd digest all those data.
3. Observe the system memory usage(2,8 GB) and influxd memory usage(0.9 GB).
4. Query metrics by curl --unix-socket /var/run/influxdb/influxdb.sock -G 'http://localhost/query?db=stress' --data-urlencode "q=SELECT * FROM "test" where time > '2021-06-02T00:00:00Z' and time < '2021-08-02T00:00:00Z' order by time desc limit 50 offset 0" -H 'Accept:application/csv'.
5. Query metrics by curl --unix-socket /var/run/influxdb/influxdb.sock -G 'http://localhost/query?db=stress' --data-urlencode "q=SELECT * FROM "test" where time > '2021-06-02T00:00:00Z' and time < '2021-08-02T00:00:00Z' order by time desc limit 50 offset 50" -H 'Accept:application/csv'.
6. Query metrics by curl --unix-socket /var/run/influxdb/influxdb.sock -G 'http://localhost/query?db=stress' --data-urlencode "q=SELECT * FROM "test" where time > '2021-06-02T00:00:00Z' and time < '2021-08-02T00:00:00Z' order by time desc limit 50 offset 1000" -H 'Accept:application/csv'. The operation is operated for 6 times.
7. Observe the system memory usage(4.x GB) and influxd memory usage(1.9 GB).
8. After 12 hours, observe again. Observer the system memory usage(4.x GB) and influxd memory usage(1.9 GB). During the time, I don't do any query operation for db stress.

__Expected behavior:__
The memory usage for influxd should not be remaining memory high. I mean G.C should happens to reduce the memory usage of influxd.

__Actual behavior:__
The memory usage for influxd is remaining memory high and doesn't release memory. If I create testing data for the past 12 months, OOM happens while performing queries. curl --unix-socket /var/run/influxdb/influxdb.sock -G 'http://localhost/query?db=stress' --data-urlencode "q=SELECT * FROM "test" order by time desc limit 50 offset 50" -H 'Accept:application/csv' . (System memory is 8 GB).

What I expected is influxd can release unused memory, but it seems like the memory is kept and doesn't release.

__Environment info:__

* System info: Run `uname -srm` and copy the output here.
Linux 4.14.24 x86_64
* InfluxDB version: Run `influxd version` and copy the output here.
InfluxDB v1.8.3 (git: 1.8 563e6c3d1a7a2790763c6289501095dbec19244e)
* Other relevant environment details: Container runtime, disk info, etc
Use alpine:3.12 as dockerfile to build docker image.

__Config:__
Copy any non-default config values here or attach the full config as a gist or file.
index-version = "tsi1"

__Logs:__
Include snippet of errors in log.

__Performance:__
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.

```sh
# Commands should be run when the bug is actively happening.
# Note: This command will run for ~30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=30s"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz` and `iostat.txt` output files.
```
[profiles.tar.gz](https://github.com/influxdata/influxdb/files/6898906/profiles.tar.gz)
[iostat.txt](https://github.com/influxdata/influxdb/files/6898907/iostat.txt)

Start by reproducing the reported memory growth with influx-stress, the stated InfluxDB configuration, and the sample curl queries. Review the attached profiles.tar.gz and iostat.txt, and run the listed profiling commands during the problem; done means repeated queries no longer leave influxd at elevated memory usage or lead to OOM.

Written by the indexing model from the issue text.