panic: keys must be added in sorted order on Influxdb v2.1

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

1. ...systemctl start influxdb
2. ...
3. ...

__Expected behavior:__
Influxdb start without restart loop.

__Actual behavior:__
Keep showing that Influxdb is not ready and throw panic saying that keys must be added in sorted order and then restart.

I have tried to remove all index and tsi files and rebuild it with command:
influxd inspect build-tsi

but didn't resolve the issue.

__Environment info:__

* System info: Linux 4.18.0-305.25.1.el8_4.x86_64 x86_64
* InfluxDB version: InfluxDB 2.1.1 (git: 657e1839de) build_date: 2021-11-09T03:03:48Z
* Other relevant environment details: No container, directly installed on Linux.

__Config:__
Used default configuration

__Logs:__
```
May 26 15:43:54 influxd-systemd-start.sh[357072]: ts=2022-05-26T19:43:54.230027Z lvl=info msg="Compacting file" log_id=0ahUBqVG000 service=storage-engine engine=tsm1 tsm1_strategy=full tsm1_optimize=false op_name=tsm1_compact_group tsm1_index=9 tsm1_file=/var/lib/influxdb/engine/data/e6c572d9ae49dccd/autogen/12/000000403-000000001.tsm
May 26 15:48:35 influxd-systemd-start.sh[357072]: ts=2022-05-26T19:48:35.981884Z lvl=info msg="TSM compaction (end)" log_id=0ahUBqVG000 service=storage-engine engine=tsm1 tsm1_strategy=full tsm1_optimize=false op_name=tsm1_compact_group op_event=end op_elapsed=281751.905ms
May 26 15:48:35 influxd-systemd-start.sh[357072]: panic: keys must be added in sorted order: TMPWWILDCARD_SYML,mfile=CTP_SAT1,symbol=xCSYLlLaGJ5#!~#Eng < TMP_WILDCARD_SYML,mfile=CTP_SAT1,symbol=xCSXNcQsIa7#!~#source
May 26 15:48:35 influxd-systemd-start.sh[357072]: goroutine 1629 [running]:
May 26 15:48:35 influxd-systemd-start.sh[357072]: github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*directIndex).Add(0xc02589c660, {0x7f6c853ae404, 0x4, 0xc002e291f0}, 0x3, 0x16f08e9bd9f5ff66, 0x16f08e9bd9f5ff66, 0x1f18f0a4, 0x16)
May 26 15:48:35 influxd-systemd-start.sh[357072]: /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/writer.go:337 +0x488
May 26 15:48:35 influxd-systemd-start.sh[357072]: github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*tsmWriter).WriteBlock(0xc0096d4f80, {0x7f6c853ae404, 0x3a, 0x933e990}, 0x2, 0x2, {0x7f6c2c8090a8, 0x0, 0x61ee3cec})
May 26 15:48:35 influxd-systemd-start.sh[357072]: /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/writer.go:686 +0x18e
May 26 15:48:35 influxd-systemd-start.sh[357072]: github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Compactor).write(0xc0006ba090, {0xc000f3e6c0, 0x55}, {0x40b51f8, 0xc000145760}, 0x1, 0x20300b)
May 26 15:48:35 influxd-systemd-start.sh[357072]: /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/compact.go:1178 +0x452
May 26 15:48:35 influxd-systemd-start.sh[357072]: github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Compactor).writeNewFiles(0xc0006ba090, 0x10, 0x3e8, {0xc0009b7200, 0xa, 0xc000f3e780}, {0x40b51f8, 0xc000145760}, 0x10, 0xc0014d58c0)
May 26 15:48:35 influxd-systemd-start.sh[357072]: /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/compact.go:1059 +0x35e
May 26 15:48:35 influxd-systemd-start.sh[357072]: github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Compactor).compact(0xc0006ba090, 0x0, {0xc0009b7200, 0x7f6d02f2c5b8, 0x10}, 0xc01047c800)
May 26 15:48:35 influxd-systemd-start.sh[357072]: /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/compact.go:966 +0x285
May 26 15:48:35 influxd-systemd-start.sh[357072]: github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Compactor).CompactFull(0xc0006ba090, {0xc0009b7200, 0xa, 0x10}, 0x2)
May 26 15:48:35 influxd-systemd-start.sh[357072]: /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/compact.go:984 +0x1d0
May 26 15:48:35 influxd-systemd-start.sh[357072]: github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*compactionStrategy).compactGroup(0xc0005da230)
May 26 15:48:35 influxd-systemd-start.sh[357072]: /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:2170 +0x216
May 26 15:48:35 influxd-systemd-start.sh[357072]: github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*compactionStrategy).Apply(0xc0005da230)
May 26 15:48:35 influxd-systemd-start.sh[357072]: /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:2148 +0x37
May 26 15:48:35 influxd-systemd-start.sh[357072]: github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Engine).compactFull.func1()
May 26 15:48:35 influxd-systemd-start.sh[357072]: /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:2117 +0xf0
May 26 15:48:35 influxd-systemd-start.sh[357072]: created by github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Engine).compactFull
May 26 15:48:35 influxd-systemd-start.sh[357072]: /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:2113 +0xef
May 26 15:48:36 systemd[1]: influxdb.service: Child 357073 belongs to influxdb.service.
May 26 15:48:36 systemd[1]: influxdb.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May 26 15:48:36 systemd[1]: influxdb.service: Failed with result 'exit-code'.
May 26 15:48:36 systemd[1]: influxdb.service: Changed running -> failed
May 26 15:48:36 systemd[1]: influxdb.service: Unit entered failed state.
May 26 15:48:36 systemd[1]: influxdb.service: Changed failed -> auto-restart

```
__Performance:__
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.

```sh
# Commands should be run when the bug is actively happening.
# Note: This command will run for ~30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=30s"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz` and `iostat.txt` output files.
```
[profiles.tar.gz](https://github.com/influxdata/influxdb/files/8817781/profiles.tar.gz)
[iostat.txt](https://github.com/influxdata/influxdb/files/8817782/iostat.txt)

Start with the panic path in tsdb/engine/tsm1/writer.go, then read the compaction flow in compact.go and engine.go. Reproduce the failure around systemctl start influxdb and the reported TSI rebuild command using the supplied logs and environment details. Done means compaction no longer panics on the affected data and InfluxDB starts without a restart loop.

Written by the indexing model from the issue text.