influxdata / influxdata/influxdb
2.0.4 Recorder handler error "Failed to finish run" (timeout) every 10 seconds
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
1. Create a bucket, dashboard, alert
2. Insert measurements of 27 fields every second
__Expected behavior:__
No timeouts.
__Actual behavior:__
The journal is filled with lines that a task failed to run due to timeouts .
__Regression:__
1. Restart service: no improvement
2. Reboot: no improvement
__Environment info:__
* System info: Linux 5.10.44-1-ARCH aarch64
* InfluxDB version: InfluxDB 2.0.4 (git: 4e7a59bb9a) build_date: 2021-02-08T17:47:02Z
* Other relevant environment details: /dev/mmcblk0p2 Size: 117G Used: 7.6G Free: 104G Use: 7% Mounted on: /
__Config:__
CGroup: /system.slice/influxdb.service
`-628 /usr/bin/influxd --reporting-disabled --session-length=403200 --storage-wal-fsync-delay=15m
__Logs:__
```
SYSTEMD_LESS=FRXMK journalctl /usr/bin/influxd --since "2 minutes ago"
-- Journal begins at Wed 2021-02-03 23:22:09 CET, ends at Sun 2021-06-20 17:47:00 CEST. --
Jun 20 17:45:05 influxPi4B influxd[628]: ts=2021-06-20T15:45:05.479673Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:45:06 influxPi4B influxd[628]: ts=2021-06-20T15:45:06.529897Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b6858136081000 error=timeout
Jun 20 17:45:11 influxPi4B influxd[628]: ts=2021-06-20T15:45:11.365989Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b6858618081000 error=timeout
Jun 20 17:45:15 influxPi4B influxd[628]: ts=2021-06-20T15:45:15.518750Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:45:16 influxPi4B influxd[628]: ts=2021-06-20T15:45:16.515952Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b6858afa081000 error=timeout
Jun 20 17:45:21 influxPi4B influxd[628]: ts=2021-06-20T15:45:21.470407Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b6858fdc081000 error=timeout
Jun 20 17:45:25 influxPi4B influxd[628]: ts=2021-06-20T15:45:25.248341Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b68594be081000 error=timeout
Jun 20 17:45:25 influxPi4B influxd[628]: ts=2021-06-20T15:45:25.559687Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:45:30 influxPi4B influxd[628]: ts=2021-06-20T15:45:30.428110Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b68599a0081000 error=timeout
Jun 20 17:45:35 influxPi4B influxd[628]: ts=2021-06-20T15:45:35.428973Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b6859e82081000 error=timeout
Jun 20 17:45:35 influxPi4B influxd[628]: ts=2021-06-20T15:45:35.600678Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:45:40 influxPi4B influxd[628]: ts=2021-06-20T15:45:40.295101Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685a364081000 error=timeout
Jun 20 17:45:45 influxPi4B influxd[628]: ts=2021-06-20T15:45:45.196706Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685a846081000 error=timeout
Jun 20 17:45:45 influxPi4B influxd[628]: ts=2021-06-20T15:45:45.640996Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:45:50 influxPi4B influxd[628]: ts=2021-06-20T15:45:50.201711Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685ad28081000 error=timeout
Jun 20 17:45:55 influxPi4B influxd[628]: ts=2021-06-20T15:45:55.201571Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685b20a081000 error=timeout
Jun 20 17:45:55 influxPi4B influxd[628]: ts=2021-06-20T15:45:55.680532Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:46:00 influxPi4B influxd[628]: ts=2021-06-20T15:46:00.209917Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685b6ec081000 error=timeout
Jun 20 17:46:05 influxPi4B influxd[628]: ts=2021-06-20T15:46:05.203518Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685bbce081000 error=timeout
Jun 20 17:46:05 influxPi4B influxd[628]: ts=2021-06-20T15:46:05.720882Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:46:10 influxPi4B influxd[628]: ts=2021-06-20T15:46:10.209241Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685c0b0081000 error=timeout
Jun 20 17:46:15 influxPi4B influxd[628]: ts=2021-06-20T15:46:15.752955Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:46:16 influxPi4B influxd[628]: ts=2021-06-20T15:46:16.399144Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685c592081000 error=timeout
Jun 20 17:46:21 influxPi4B influxd[628]: ts=2021-06-20T15:46:21.932915Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685ca74081000 error=timeout
Jun 20 17:46:25 influxPi4B influxd[628]: ts=2021-06-20T15:46:25.272418Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685cf56081000 error=timeout
Jun 20 17:46:25 influxPi4B influxd[628]: ts=2021-06-20T15:46:25.792914Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:46:30 influxPi4B influxd[628]: ts=2021-06-20T15:46:30.376289Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685d438081000 error=timeout
Jun 20 17:46:35 influxPi4B influxd[628]: ts=2021-06-20T15:46:35.457821Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685d91a081000 error=timeout
Jun 20 17:46:35 influxPi4B influxd[628]: ts=2021-06-20T15:46:35.832596Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:46:40 influxPi4B influxd[628]: ts=2021-06-20T15:46:40.563183Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685ddfc081000 error=timeout
Jun 20 17:46:45 influxPi4B influxd[628]: ts=2021-06-20T15:46:45.628509Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685e2de081000 error=timeout
Jun 20 17:46:45 influxPi4B influxd[628]: ts=2021-06-20T15:46:45.872642Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:46:50 influxPi4B influxd[628]: ts=2021-06-20T15:46:50.480716Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685e7c0081000 error=timeout
Jun 20 17:46:55 influxPi4B influxd[628]: ts=2021-06-20T15:46:55.573301Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685eca2081000 error=timeout
Jun 20 17:46:55 influxPi4B influxd[628]: ts=2021-06-20T15:46:55.912185Z lvl=error msg="Recorder handler error" log_id=0UrV8VlW000 error=timeout
Jun 20 17:47:00 influxPi4B influxd[628]: ts=2021-06-20T15:47:00.531191Z lvl=error msg="Failed to finish run" log_id=0UrV8VlW000 service=task-executor taskID=075463d201fd8000 runID=07b685f184081000 error=timeout
```
__Performance:__
Systemd unit file contains:
```
[Service]
User=influxdb
Group=influxdb
LimitNOFILE=65536
#ExecStart=
ExecStart=/usr/bin/influxd --reporting-disabled --session-length=403200 --storage-wal-fsync-delay=15m
KillMode=control-group
CPUWeight=20
Restart=on-failure
```
__Debug output:__
```
goroutine profile: total 621
367 @ 0x435750 0x406328 0x405ff8 0xdc5e7c 0xdc53a4 0xd87b4c 0xba0c30 0xbb2fa8 0x149ab8c 0x14a2548 0x467fc4
# 0xdc5e7b github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*WAL).writeToLog+0x23b /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/wal.go:461
# 0xdc53a3 github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*WAL).WriteMulti+0x63 /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/wal.go:316
# 0xd87b4b github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Engine).WritePoints+0xaab /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:1341
# 0xba0c2f github.com/influxdata/influxdb/v2/tsdb.(*Shard).WritePoints+0x1bf /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/shard.go:535
# 0xbb2fa7 github.com/influxdata/influxdb/v2/tsdb.(*Store).WriteToShard+0x1e7 /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/store.go:1425
# 0x149ab8b github.com/influxdata/influxdb/v2/v1/coordinator.(*PointsWriter).writeToShard+0x8b /home/circleci/go/src/github.com/influxdata/influxdb/v1/coordinator/points_writer.go:363
# 0x14a2547 github.com/influxdata/influxdb/v2/v1/coordinator.(*PointsWriter).WritePointsPrivileged.func1+0x67 /home/circleci/go/src/github.com/influxdata/influxdb/v1/coordinator/points_writer.go:306
128 @ 0x435750 0x406328 0x406038 0x19f97bc 0x467fc4
# 0x19f97bb github.com/influxdata/influxdb/v2/task/backend/scheduler.(*TreeScheduler).work+0x7b /home/circleci/go/src/github.com/influxdata/influxdb/task/backend/scheduler/treescheduler.go:312
37 @ 0x435750 0x4641cc 0x464199 0x46fad0 0x14eec04 0x467fc4
# 0x464198 sync.runtime_notifyListWait+0xc8 /usr/local/go/src/runtime/sema.go:513
# 0x46facf sync.(*Cond).Wait+0xaf /usr/local/go/src/sync/cond.go:56
# 0x14eec03 github.com/nats-io/go-nats.(*Conn).waitForMsgs+0x5b3 /home/circleci/go/pkg/mod/github.com/nats-io/go-nats@v1.7.0/nats.go:1979
10 @ 0x435750 0x4454e4 0x149a788 0x149a218 0x14a5358 0x14a71c4 0x17f592c 0x7195b0 0xa85dc8 0xa82bd8 0x17f54a4 0xa432b0 0x7195b0 0xa3fe58 0x7195b0 0x1741f88 0x7195b0 0x173c0b4 0x7195b0 0x1742aa4 0x7195b0 0x17f9d94 0x7195b0 0xa3dfb4 0x899660 0x17b9dd8 0x1741f88 0x7195b0 0x1742aa4 0x7195b0 0x17d25c8 0xa432b0
# 0x149a787 github.com/influxdata/influxdb/v2/v1/coordinator.(*PointsWriter).WritePointsPrivileged+0x547 /home/circleci/go/src/github.com/influxdata/influxdb/v1/coordinator/points_writer.go:343
# 0x149a217 github.com/influxdata/influxdb/v2/v1/coordinator.(*PointsWriter).WritePoints+0x67 /home/circleci/go/src/github.com/influxdata/influxdb/v1/coordinator/points_writer.go:281
# 0x14a5357 github.com/influxdata/influxdb/v2/storage.(*Engine).WritePoints+0x177 /home/circleci/go/src/github.com/influxdata/influxdb/storage/engine.go:258
# 0x14a71c3 github.com/influxdata/influxdb/v2/storage.(*LoggingPointsWriter).WritePoints+0x93 /home/circleci/go/src/github.com/influxdata/influxdb/storage/points_writer.go:37
# 0x17f592b github.com/influxdata/influxdb/v2/http.(*WriteHandler).handleWrite+0x46b /home/circleci/go/src/github.com/influxdata/influxdb/http/write_handler.go:190
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0xa85dc7 github.com/influxdata/httprouter.(*Router).Handler.func1+0x177 /home/circleci/go/pkg/mod/github.com/influxdata/httprouter@v1.3.1-0.20191122104820-ee83e2772f69/router.go:325
# 0xa82bd7 github.com/influxdata/httprouter.(*Router).ServeHTTP+0x807 /home/circleci/go/pkg/mod/github.com/influxdata/httprouter@v1.3.1-0.20191122104820-ee83e2772f69/router.go:453
# 0x17f54a3 github.com/influxdata/influxdb/v2/http.(*WriteHandler).ServeHTTP+0x43 /home/circleci/go/src/github.com/influxdata/influxdb/http/write_handler.go:133
# 0xa432af github.com/go-chi/chi.(*Mux).Mount.func1+0x10f /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:298
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0xa3fe57 github.com/go-chi/chi.(*Mux).routeHTTP+0x237 /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:431
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x1741f87 github.com/influxdata/influxdb/v2/kit/transport/http.SetCORS.func1+0xc7 /home/circleci/go/src/github.com/influxdata/influxdb/kit/transport/http/middleware.go:34
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x173c0b3 github.com/go-chi/chi/middleware.StripSlashes.func1+0xe3 /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/middleware/strip.go:25
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x1742aa3 github.com/influxdata/influxdb/v2/kit/transport/http.SkipOptions.func1+0xc3 /home/circleci/go/src/github.com/influxdata/influxdb/kit/transport/http/middleware.go:72
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x17f9d93 github.com/influxdata/influxdb/v2/http.panicMW.func1.1+0x93 /home/circleci/go/src/github.com/influxdata/influxdb/http/router.go:126
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0xa3dfb3 github.com/go-chi/chi.(*Mux).ServeHTTP+0x493 /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:70
# 0x89965f github.com/influxdata/influxdb/v2/kit/feature.(*Handler).ServeHTTP+0x1cf /home/circleci/go/src/github.com/influxdata/influxdb/kit/feature/middleware.go:43
# 0x17b9dd7 github.com/influxdata/influxdb/v2/http.(*AuthenticationHandler).ServeHTTP+0x247 /home/circleci/go/src/github.com/influxdata/influxdb/http/authentication_middleware.go:125
# 0x1741f87 github.com/influxdata/influxdb/v2/kit/transport/http.SetCORS.func1+0xc7 /home/circleci/go/src/github.com/influxdata/influxdb/kit/transport/http/middleware.go:34
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x1742aa3 github.com/influxdata/influxdb/v2/kit/transport/http.SkipOptions.func1+0xc3 /home/circleci/go/src/github.com/influxdata/influxdb/kit/transport/http/middleware.go:72
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x17d25c7 github.com/influxdata/influxdb/v2/http.(*PlatformHandler).ServeHTTP+0x177 /home/circleci/go/src/github.com/influxdata/influxdb/http/platform_handler.go:78
# 0xa432af github.com/go-chi/chi.(*Mux).Mount.func1+0x10f /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:298
10 @ 0x435750 0x4454e4 0x149a788 0x149a218 0x14a5358 0x14a73f0 0x17f592c 0x7195b0 0xa85dc8 0xa82bd8 0x17f54a4 0xa432b0 0x7195b0 0xa3fe58 0x7195b0 0x1741f88 0x7195b0 0x173c0b4 0x7195b0 0x1742aa4 0x7195b0 0x17f9d94 0x7195b0 0xa3dfb4 0x899660 0x17b9dd8 0x1741f88 0x7195b0 0x1742aa4 0x7195b0 0x17d25c8 0xa432b0
# 0x149a787 github.com/influxdata/influxdb/v2/v1/coordinator.(*PointsWriter).WritePointsPrivileged+0x547 /home/circleci/go/src/github.com/influxdata/influxdb/v1/coordinator/points_writer.go:343
# 0x149a217 github.com/influxdata/influxdb/v2/v1/coordinator.(*PointsWriter).WritePoints+0x67 /home/circleci/go/src/github.com/influxdata/influxdb/v1/coordinator/points_writer.go:281
# 0x14a5357 github.com/influxdata/influxdb/v2/storage.(*Engine).WritePoints+0x177 /home/circleci/go/src/github.com/influxdata/influxdb/storage/engine.go:258
# 0x14a73ef github.com/influxdata/influxdb/v2/storage.(*LoggingPointsWriter).WritePoints+0x2bf /home/circleci/go/src/github.com/influxdata/influxdb/storage/points_writer.go:63
# 0x17f592b github.com/influxdata/influxdb/v2/http.(*WriteHandler).handleWrite+0x46b /home/circleci/go/src/github.com/influxdata/influxdb/http/write_handler.go:190
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0xa85dc7 github.com/influxdata/httprouter.(*Router).Handler.func1+0x177 /home/circleci/go/pkg/mod/github.com/influxdata/httprouter@v1.3.1-0.20191122104820-ee83e2772f69/router.go:325
# 0xa82bd7 github.com/influxdata/httprouter.(*Router).ServeHTTP+0x807 /home/circleci/go/pkg/mod/github.com/influxdata/httprouter@v1.3.1-0.20191122104820-ee83e2772f69/router.go:453
# 0x17f54a3 github.com/influxdata/influxdb/v2/http.(*WriteHandler).ServeHTTP+0x43 /home/circleci/go/src/github.com/influxdata/influxdb/http/write_handler.go:133
# 0xa432af github.com/go-chi/chi.(*Mux).Mount.func1+0x10f /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:298
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0xa3fe57 github.com/go-chi/chi.(*Mux).routeHTTP+0x237 /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:431
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x1741f87 github.com/influxdata/influxdb/v2/kit/transport/http.SetCORS.func1+0xc7 /home/circleci/go/src/github.com/influxdata/influxdb/kit/transport/http/middleware.go:34
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x173c0b3 github.com/go-chi/chi/middleware.StripSlashes.func1+0xe3 /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/middleware/strip.go:25
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x1742aa3 github.com/influxdata/influxdb/v2/kit/transport/http.SkipOptions.func1+0xc3 /home/circleci/go/src/github.com/influxdata/influxdb/kit/transport/http/middleware.go:72
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x17f9d93 github.com/influxdata/influxdb/v2/http.panicMW.func1.1+0x93 /home/circleci/go/src/github.com/influxdata/influxdb/http/router.go:126
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0xa3dfb3 github.com/go-chi/chi.(*Mux).ServeHTTP+0x493 /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:70
# 0x89965f github.com/influxdata/influxdb/v2/kit/feature.(*Handler).ServeHTTP+0x1cf /home/circleci/go/src/github.com/influxdata/influxdb/kit/feature/middleware.go:43
# 0x17b9dd7 github.com/influxdata/influxdb/v2/http.(*AuthenticationHandler).ServeHTTP+0x247 /home/circleci/go/src/github.com/influxdata/influxdb/http/authentication_middleware.go:125
# 0x1741f87 github.com/influxdata/influxdb/v2/kit/transport/http.SetCORS.func1+0xc7 /home/circleci/go/src/github.com/influxdata/influxdb/kit/transport/http/middleware.go:34
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x1742aa3 github.com/influxdata/influxdb/v2/kit/transport/http.SkipOptions.func1+0xc3 /home/circleci/go/src/github.com/influxdata/influxdb/kit/transport/http/middleware.go:72
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x17d25c7 github.com/influxdata/influxdb/v2/http.(*PlatformHandler).ServeHTTP+0x177 /home/circleci/go/src/github.com/influxdata/influxdb/http/platform_handler.go:78
# 0xa432af github.com/go-chi/chi.(*Mux).Mount.func1+0x10f /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:298
10 @ 0x435750 0x4454e4 0x19658c8 0x1968e84 0x467fc4
# 0x19658c7 github.com/influxdata/influxdb/v2/query/control.(*Controller).processQueryQueue+0x97 /home/circleci/go/src/github.com/influxdata/influxdb/query/control/controller.go:401
# 0x1968e83 github.com/influxdata/influxdb/v2/query/control.New.func1+0x53 /home/circleci/go/src/github.com/influxdata/influxdb/query/control/controller.go:208
5 @ 0x435750 0x406328 0x406038 0x14ef878 0x467fc4
# 0x14ef877 github.com/nats-io/go-nats.(*Conn).flusher+0xe7 /home/circleci/go/pkg/mod/github.com/nats-io/go-nats@v1.7.0/nats.go:2184
5 @ 0x435750 0x42ee34 0x462694 0x4d44f8 0x4d5474 0x4d5459 0x62f4f4 0x640514 0x14ee364 0x467fc4
# 0x462693 internal/poll.runtime_pollWait+0x43 /usr/local/go/src/runtime/netpoll.go:220
# 0x4d44f7 internal/poll.(*pollDesc).wait+0x37 /usr/local/go/src/internal/poll/fd_poll_runtime.go:87
# 0x4d5473 internal/poll.(*pollDesc).waitRead+0x163 /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
# 0x4d5458 internal/poll.(*FD).Read+0x148 /usr/local/go/src/internal/poll/fd_unix.go:159
# 0x62f4f3 net.(*netFD).Read+0x43 /usr/local/go/src/net/fd_posix.go:55
# 0x640513 net.(*conn).Read+0x73 /usr/local/go/src/net/net.go:182
# 0x14ee363 github.com/nats-io/go-nats.(*Conn).readLoop+0x193 /home/circleci/go/pkg/mod/github.com/nats-io/go-nats@v1.7.0/nats.go:1942
5 @ 0x435750 0x42ee34 0x462694 0x4d44f8 0x4d5474 0x4d5459 0x62f4f4 0x640514 0x151e914 0x467fc4
# 0x462693 internal/poll.runtime_pollWait+0x43 /usr/local/go/src/runtime/netpoll.go:220
# 0x4d44f7 internal/poll.(*pollDesc).wait+0x37 /usr/local/go/src/internal/poll/fd_poll_runtime.go:87
# 0x4d5473 internal/poll.(*pollDesc).waitRead+0x163 /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
# 0x4d5458 internal/poll.(*FD).Read+0x148 /usr/local/go/src/internal/poll/fd_unix.go:159
# 0x62f4f3 net.(*netFD).Read+0x43 /usr/local/go/src/net/fd_posix.go:55
# 0x640513 net.(*conn).Read+0x73 /usr/local/go/src/net/net.go:182
# 0x151e913 github.com/nats-io/gnatsd/server.(*client).readLoop+0x3b3 /home/circleci/go/pkg/mod/github.com/nats-io/gnatsd@v1.3.0/server/client.go:420
5 @ 0x435750 0x4641cc 0x464199 0x46fad0 0x14ede80 0x467fc4
# 0x464198 sync.runtime_notifyListWait+0xc8 /usr/local/go/src/runtime/sema.go:513
# 0x46facf sync.(*Cond).Wait+0xaf /usr/local/go/src/sync/cond.go:56
# 0x14ede7f github.com/nats-io/go-nats.(*asyncCallbacksHandler).asyncCBDispatcher+0x2f /home/circleci/go/pkg/mod/github.com/nats-io/go-nats@v1.7.0/nats.go:1857
5 @ 0x435750 0x4641cc 0x464199 0x46fad0 0x151e504 0x467fc4
# 0x464198 sync.runtime_notifyListWait+0xc8 /usr/local/go/src/runtime/sema.go:513
# 0x46facf sync.(*Cond).Wait+0xaf /usr/local/go/src/sync/cond.go:56
# 0x151e503 github.com/nats-io/gnatsd/server.(*client).writeLoop+0x163 /home/circleci/go/pkg/mod/github.com/nats-io/gnatsd@v1.3.0/server/client.go:386
3 @ 0x435750 0x4454e4 0xd8a91c 0xdd18f0 0x467fc4
# 0xd8a91b github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Engine).compactCache+0x11b /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:1944
# 0xdd18ef github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Engine).enableSnapshotCompactions.func1+0x4f /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:508
3 @ 0x435750 0x4454e4 0xd8b0f4 0xdd1864 0x467fc4
# 0xd8b0f3 github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Engine).compact+0x1f3 /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:1991
# 0xdd1863 github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*Engine).enableLevelCompactions.func1+0x53 /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:429
3 @ 0x435750 0x4454e4 0xdd9a04 0x467fc4
# 0xdd9a03 github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*WAL).scheduleSync.func1+0xe3 /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/wal.go:279
2 @ 0x435750 0x42ee34 0x462694 0x4d44f8 0x4d5474 0x4d5459 0x62f4f4 0x640514 0x713050 0x467fc4
# 0x462693 internal/poll.runtime_pollWait+0x43 /usr/local/go/src/runtime/netpoll.go:220
# 0x4d44f7 internal/poll.(*pollDesc).wait+0x37 /usr/local/go/src/internal/poll/fd_poll_runtime.go:87
# 0x4d5473 internal/poll.(*pollDesc).waitRead+0x163 /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
# 0x4d5458 internal/poll.(*FD).Read+0x148 /usr/local/go/src/internal/poll/fd_unix.go:159
# 0x62f4f3 net.(*netFD).Read+0x43 /usr/local/go/src/net/fd_posix.go:55
# 0x640513 net.(*conn).Read+0x73 /usr/local/go/src/net/net.go:182
# 0x71304f net/http.(*connReader).backgroundRead+0x4f /usr/local/go/src/net/http/server.go:690
2 @ 0x435750 0x4454e4 0x149a788 0x149a218 0x14a5358 0x1a008d0 0x19fbcd8 0x1a05840 0x1a06778 0x1a04dfc 0x1a0892c 0x467fc4
# 0x149a787 github.com/influxdata/influxdb/v2/v1/coordinator.(*PointsWriter).WritePointsPrivileged+0x547 /home/circleci/go/src/github.com/influxdata/influxdb/v1/coordinator/points_writer.go:343
# 0x149a217 github.com/influxdata/influxdb/v2/v1/coordinator.(*PointsWriter).WritePoints+0x67 /home/circleci/go/src/github.com/influxdata/influxdb/v1/coordinator/points_writer.go:281
# 0x14a5357 github.com/influxdata/influxdb/v2/storage.(*Engine).WritePoints+0x177 /home/circleci/go/src/github.com/influxdata/influxdb/storage/engine.go:258
# 0x1a008cf github.com/influxdata/influxdb/v2/task/backend.(*StoragePointsWriterRecorder).Record+0x8cf /home/circleci/go/src/github.com/influxdata/influxdb/task/backend/run_recorder.go:69
# 0x19fbcd7 github.com/influxdata/influxdb/v2/task/backend.(*AnalyticalStorage).FinishRun+0x1d7 /home/circleci/go/src/github.com/influxdata/influxdb/task/backend/analytical_storage.go:84
# 0x1a0583f github.com/influxdata/influxdb/v2/task/backend/executor.(*worker).finish+0x63f /home/circleci/go/src/github.com/influxdata/influxdb/task/backend/executor/executor.go:482
# 0x1a06777 github.com/influxdata/influxdb/v2/task/backend/executor.(*worker).executeQuery+0x887 /home/circleci/go/src/github.com/influxdata/influxdb/task/backend/executor/executor.go:550
# 0x1a04dfb github.com/influxdata/influxdb/v2/task/backend/executor.(*worker).work+0x40b /home/circleci/go/src/github.com/influxdata/influxdb/task/backend/executor/executor.go:421
# 0x1a0892b github.com/influxdata/influxdb/v2/task/backend/executor.(*Executor).startWorker.func1+0x7b /home/circleci/go/src/github.com/influxdata/influxdb/task/backend/executor/executor.go:291
1 @ 0x40b2f0 0x4647a4 0x14c8720 0x467fc4
# 0x4647a3 os/signal.signal_recv+0xc3 /usr/local/go/src/runtime/sigqueue.go:147
# 0x14c871f os/signal.loop+0x1f /usr/local/go/src/os/signal/signal_unix.go:23
1 @ 0x435750 0x406328 0x405ff8 0x1a490a8 0xc9b67c 0xbf3b3c 0xbf43b8 0x1a67cc4 0x1a67cc9 0x43537c 0x467fc4
# 0x1a490a7 github.com/influxdata/influxdb/v2/cmd/influxd/launcher.cmdRunE.func1+0x2d7 /home/circleci/go/src/github.com/influxdata/influxdb/cmd/influxd/launcher/cmd.go:97
# 0xc9b67b github.com/influxdata/influxdb/v2/kit/cli.NewCommand.func1+0x2b /home/circleci/go/src/github.com/influxdata/influxdb/kit/cli/viper.go:54
# 0xbf3b3b github.com/spf13/cobra.(*Command).execute+0x30b /home/circleci/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:842
# 0xbf43b7 github.com/spf13/cobra.(*Command).ExecuteC+0x267 /home/circleci/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950
# 0x1a67cc3 github.com/spf13/cobra.(*Command).Execute+0x333 /home/circleci/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
# 0x1a67cc8 main.main+0x338 /home/circleci/go/src/github.com/influxdata/influxdb/cmd/influxd/main.go:43
# 0x43537b runtime.main+0x20b /usr/local/go/src/runtime/proc.go:204
1 @ 0x435750 0x406328 0x406038 0xe27e38 0x467fc4
# 0xe27e37 github.com/influxdata/influxdb/v2/bolt.init.0.func1+0x77 /home/circleci/go/src/github.com/influxdata/influxdb/bolt/metrics.go:115
1 @ 0x435750 0x406328 0x406038 0xe8a344 0x467fc4
# 0xe8a343 github.com/influxdata/flux/execute.createOperatorProfiler.func1+0xa3 /home/circleci/go/pkg/mod/github.com/influxdata/flux@v0.104.0/execute/profiler.go:108
1 @ 0x435750 0x42ee34 0x462694 0x4d44f8 0x4d5474 0x4d5459 0x62f4f4 0x640514 0x713664 0x502768 0x5034b8 0x5036e0 0x6aeb00 0x70e624 0x70e645 0x714810 0x7184cc 0x467fc4
# 0x462693 internal/poll.runtime_pollWait+0x43 /usr/local/go/src/runtime/netpoll.go:220
# 0x4d44f7 internal/poll.(*pollDesc).wait+0x37 /usr/local/go/src/internal/poll/fd_poll_runtime.go:87
# 0x4d5473 internal/poll.(*pollDesc).waitRead+0x163 /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
# 0x4d5458 internal/poll.(*FD).Read+0x148 /usr/local/go/src/internal/poll/fd_unix.go:159
# 0x62f4f3 net.(*netFD).Read+0x43 /usr/local/go/src/net/fd_posix.go:55
# 0x640513 net.(*conn).Read+0x73 /usr/local/go/src/net/net.go:182
# 0x713663 net/http.(*connReader).Read+0x243 /usr/local/go/src/net/http/server.go:798
# 0x502767 bufio.(*Reader).fill+0xf7 /usr/local/go/src/bufio/bufio.go:101
# 0x5034b7 bufio.(*Reader).ReadSlice+0x37 /usr/local/go/src/bufio/bufio.go:360
# 0x5036df bufio.(*Reader).ReadLine+0x2f /usr/local/go/src/bufio/bufio.go:389
# 0x6aeaff net/textproto.(*Reader).readLineSlice+0x5f /usr/local/go/src/net/textproto/reader.go:58
# 0x70e623 net/textproto.(*Reader).ReadLine+0x73 /usr/local/go/src/net/textproto/reader.go:39
# 0x70e644 net/http.readRequest+0x94 /usr/local/go/src/net/http/request.go:1012
# 0x71480f net/http.(*conn).readRequest+0x14f /usr/local/go/src/net/http/server.go:984
# 0x7184cb net/http.(*conn).serve+0x5ab /usr/local/go/src/net/http/server.go:1851
1 @ 0x435750 0x42ee34 0x462694 0x4d44f8 0x4d5474 0x4d5459 0x62f4f4 0x640514 0x72f960 0x502768 0x5028e0 0x730550 0x467fc4
# 0x462693 internal/poll.runtime_pollWait+0x43 /usr/local/go/src/runtime/netpoll.go:220
# 0x4d44f7 internal/poll.(*pollDesc).wait+0x37 /usr/local/go/src/internal/poll/fd_poll_runtime.go:87
# 0x4d5473 internal/poll.(*pollDesc).waitRead+0x163 /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
# 0x4d5458 internal/poll.(*FD).Read+0x148 /usr/local/go/src/internal/poll/fd_unix.go:159
# 0x62f4f3 net.(*netFD).Read+0x43 /usr/local/go/src/net/fd_posix.go:55
# 0x640513 net.(*conn).Read+0x73 /usr/local/go/src/net/net.go:182
# 0x72f95f net/http.(*persistConn).Read+0x5f /usr/local/go/src/net/http/transport.go:1887
# 0x502767 bufio.(*Reader).fill+0xf7 /usr/local/go/src/bufio/bufio.go:101
# 0x5028df bufio.(*Reader).Peek+0x3f /usr/local/go/src/bufio/bufio.go:139
# 0x73054f net/http.(*persistConn).readLoop+0x16f /usr/local/go/src/net/http/transport.go:2040
1 @ 0x435750 0x42ee34 0x462694 0x4d44f8 0x4d6ab0 0x4d6a95 0x63075c 0x64966c 0x6485ec 0x1544604 0x1543748 0x467fc4
# 0x462693 internal/poll.runtime_pollWait+0x43 /usr/local/go/src/runtime/netpoll.go:220
# 0x4d44f7 internal/poll.(*pollDesc).wait+0x37 /usr/local/go/src/internal/poll/fd_poll_runtime.go:87
# 0x4d6aaf internal/poll.(*pollDesc).waitRead+0x19f /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
# 0x4d6a94 internal/poll.(*FD).Accept+0x184 /usr/local/go/src/internal/poll/fd_unix.go:394
# 0x63075b net.(*netFD).accept+0x2b /usr/local/go/src/net/fd_unix.go:172
# 0x64966b net.(*TCPListener).accept+0x2b /usr/local/go/src/net/tcpsock_posix.go:139
# 0x6485eb net.(*TCPListener).Accept+0x5b /usr/local/go/src/net/tcpsock.go:261
# 0x1544603 github.com/nats-io/gnatsd/server.(*Server).AcceptLoop+0x583 /home/circleci/go/pkg/mod/github.com/nats-io/gnatsd@v1.3.0/server/server.go:488
# 0x1543747 github.com/nats-io/gnatsd/server.(*Server).Start+0x377 /home/circleci/go/pkg/mod/github.com/nats-io/gnatsd@v1.3.0/server/server.go:329
1 @ 0x435750 0x42ee34 0x462694 0x4d44f8 0x4d6ab0 0x4d6a95 0x63075c 0x64966c 0x6485ec 0x71c678 0x1a49e20 0x467fc4
# 0x462693 internal/poll.runtime_pollWait+0x43 /usr/local/go/src/runtime/netpoll.go:220
# 0x4d44f7 internal/poll.(*pollDesc).wait+0x37 /usr/local/go/src/internal/poll/fd_poll_runtime.go:87
# 0x4d6aaf internal/poll.(*pollDesc).waitRead+0x19f /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
# 0x4d6a94 internal/poll.(*FD).Accept+0x184 /usr/local/go/src/internal/poll/fd_unix.go:394
# 0x63075b net.(*netFD).accept+0x2b /usr/local/go/src/net/fd_unix.go:172
# 0x64966b net.(*TCPListener).accept+0x2b /usr/local/go/src/net/tcpsock_posix.go:139
# 0x6485eb net.(*TCPListener).Accept+0x5b /usr/local/go/src/net/tcpsock.go:261
# 0x71c677 net/http.(*Server).Serve+0x217 /usr/local/go/src/net/http/server.go:2937
# 0x1a49e1f github.com/influxdata/influxdb/v2/cmd/influxd/launcher.(*Launcher).run.func4+0x53f /home/circleci/go/src/github.com/influxdata/influxdb/cmd/influxd/launcher/launcher.go:1009
1 @ 0x435750 0x4454e4 0x1463e1c 0x467fc4
# 0x1463e1b github.com/influxdata/influxdb/v2/v1/services/precreator.(*Service).runPrecreation+0xdb /home/circleci/go/src/github.com/influxdata/influxdb/v1/services/precreator/service.go:77
1 @ 0x435750 0x4454e4 0x1482e88 0x1483b54 0x467fc4
# 0x1482e87 github.com/influxdata/influxdb/v2/v1/services/retention.(*Service).run+0xa07 /home/circleci/go/src/github.com/influxdata/influxdb/v1/services/retention/service.go:85
# 0x1483b53 github.com/influxdata/influxdb/v2/v1/services/retention.(*Service).Open.func1+0x63 /home/circleci/go/src/github.com/influxdata/influxdb/v1/services/retention/service.go:55
1 @ 0x435750 0x4454e4 0x149a788 0x149a218 0x14a5358 0x1842920 0x1842bf4 0x16222c8 0x150ec70 0x14eeb10 0x467fc4
# 0x149a787 github.com/influxdata/influxdb/v2/v1/coordinator.(*PointsWriter).WritePointsPrivileged+0x547 /home/circleci/go/src/github.com/influxdata/influxdb/v1/coordinator/points_writer.go:343
# 0x149a217 github.com/influxdata/influxdb/v2/v1/coordinator.(*PointsWriter).WritePoints+0x67 /home/circleci/go/src/github.com/influxdata/influxdb/v1/coordinator/points_writer.go:281
# 0x14a5357 github.com/influxdata/influxdb/v2/storage.(*Engine).WritePoints+0x177 /home/circleci/go/src/github.com/influxdata/influxdb/storage/engine.go:258
# 0x184291f github.com/influxdata/influxdb/v2/gather.PointWriter.Record+0xaf /home/circleci/go/src/github.com/influxdata/influxdb/gather/recorder.go:24
# 0x1842bf3 github.com/influxdata/influxdb/v2/gather.(*RecorderHandler).Process+0x2a3 /home/circleci/go/src/github.com/influxdata/influxdb/gather/recorder.go:55
# 0x16222c7 github.com/influxdata/influxdb/v2/nats.(*messageHandler).handle+0xb7 /home/circleci/go/src/github.com/influxdata/influxdb/nats/subscriber.go:38
# 0x150ec6f github.com/nats-io/go-nats-streaming.(*conn).processMsg+0x23f /home/circleci/go/pkg/mod/github.com/nats-io/go-nats-streaming@v0.4.0/stan.go:751
# 0x14eeb0f github.com/nats-io/go-nats.(*Conn).waitForMsgs+0x4bf /home/circleci/go/pkg/mod/github.com/nats-io/go-nats@v1.7.0/nats.go:2012
1 @ 0x435750 0x4454e4 0x14c8b1c 0x467fc4
# 0x14c8b1b github.com/influxdata/influxdb/v2/kit/signals.WithSignals.func1+0x9b /home/circleci/go/src/github.com/influxdata/influxdb/kit/signals/context.go:18
1 @ 0x435750 0x4454e4 0x1611124 0x467fc4
# 0x1611123 github.com/nats-io/nats-streaming-server/server.(*StanServer).ioLoop+0x5d3 /home/circleci/go/pkg/mod/github.com/influxdata/nats-streaming-server@v0.11.3-0.20201112040610-c277f7560803/server/server.go:3694
1 @ 0x435750 0x4454e4 0x1616074 0x467fc4
# 0x1616073 github.com/nats-io/nats-streaming-server/server.(*StanServer).processSubscriptionsStart+0xb3 /home/circleci/go/pkg/mod/github.com/influxdata/nats-streaming-server@v0.11.3-0.20201112040610-c277f7560803/server/server.go:4635
1 @ 0x435750 0x4454e4 0x1843148 0x1843078 0x1a4971c 0x467fc4
# 0x1843147 github.com/influxdata/influxdb/v2/gather.(*Scheduler).run+0xa7 /home/circleci/go/src/github.com/influxdata/influxdb/gather/scheduler.go:95
# 0x1843077 github.com/influxdata/influxdb/v2/gather.(*Scheduler).Run+0x67 /home/circleci/go/src/github.com/influxdata/influxdb/gather/scheduler.go:90
# 0x1a4971b github.com/influxdata/influxdb/v2/cmd/influxd/launcher.(*Launcher).run.func3+0x13b /home/circleci/go/src/github.com/influxdata/influxdb/cmd/influxd/launcher/launcher.go:614
1 @ 0x435750 0x4454e4 0x184393c 0x467fc4
# 0x184393b github.com/influxdata/influxdb/v2/gather.(*Scheduler).Run.func1+0xbb /home/circleci/go/src/github.com/influxdata/influxdb/gather/scheduler.go:82
1 @ 0x435750 0x4454e4 0x19fa828 0x467fc4
# 0x19fa827 github.com/influxdata/influxdb/v2/task/backend/scheduler.NewScheduler.func2+0x3b7 /home/circleci/go/src/github.com/influxdata/influxdb/task/backend/scheduler/treescheduler.go:156
1 @ 0x435750 0x4454e4 0x731c5c 0x467fc4
# 0x731c5b net/http.(*persistConn).writeLoop+0xbb /usr/local/go/src/net/http/transport.go:2340
1 @ 0x435750 0x4454e4 0xbb60b8 0xbb9734 0x467fc4
# 0xbb60b7 github.com/influxdata/influxdb/v2/tsdb.(*Store).monitorShards+0x337 /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/store.go:1920
# 0xbb9733 github.com/influxdata/influxdb/v2/tsdb.(*Store).Open.func1+0x43 /home/circleci/go/src/github.com/influxdata/influxdb/tsdb/store.go:228
1 @ 0x435750 0x4454e4 0xf8d7f4 0x467fc4
# 0xf8d7f3 go.opencensus.io/stats/view.(*worker).start+0xb3 /home/circleci/go/pkg/mod/go.opencensus.io@v0.22.3/stats/view/worker.go:154
1 @ 0x4622b0 0x74d708 0x74d510 0x74aae4 0x756278 0x7575e4 0x7195b0 0x71af80 0xa432b0 0x7195b0 0x1742840 0x7195b0 0xa3d36c 0xa3fe58 0x7195b0 0xa3dd60 0x17c575c 0x71c32c 0x718614 0x467fc4
# 0x4622af runtime/pprof.runtime_goroutineProfileWithLabels+0x4f /usr/local/go/src/runtime/mprof.go:716
# 0x74d707 runtime/pprof.writeRuntimeProfile+0xb7 /usr/local/go/src/runtime/pprof/pprof.go:724
# 0x74d50f runtime/pprof.writeGoroutine+0x8f /usr/local/go/src/runtime/pprof/pprof.go:684
# 0x74aae3 runtime/pprof.(*Profile).WriteTo+0x343 /usr/local/go/src/runtime/pprof/pprof.go:331
# 0x756277 net/http/pprof.handler.ServeHTTP+0x2d7 /usr/local/go/src/net/http/pprof/pprof.go:256
# 0x7575e3 net/http/pprof.Index+0x713 /usr/local/go/src/net/http/pprof/pprof.go:367
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x71af7f net/http.(*ServeMux).ServeHTTP+0x18f /usr/local/go/src/net/http/server.go:2417
# 0xa432af github.com/go-chi/chi.(*Mux).Mount.func1+0x10f /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:298
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0x174283f github.com/influxdata/influxdb/v2/kit/transport/http.Metrics.func1.1+0x12f /home/circleci/go/src/github.com/influxdata/influxdb/kit/transport/http/middleware.go:57
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0xa3d36b github.com/go-chi/chi.(*ChainHandler).ServeHTTP+0x4b /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/chain.go:31
# 0xa3fe57 github.com/go-chi/chi.(*Mux).routeHTTP+0x237 /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:431
# 0x7195af net/http.HandlerFunc.ServeHTTP+0x3f /usr/local/go/src/net/http/server.go:2042
# 0xa3dd5f github.com/go-chi/chi.(*Mux).ServeHTTP+0x23f /home/circleci/go/pkg/mod/github.com/go-chi/chi@v4.1.0+incompatible/mux.go:86
# 0x17c575b github.com/influxdata/influxdb/v2/http.(*Handler).ServeHTTP+0x4b /home/circleci/go/src/github.com/influxdata/influxdb/http/handler.go:143
# 0x71c32b net/http.serverHandler.ServeHTTP+0xbb /usr/local/go/src/net/http/server.go:2843
# 0x718613 net/http.(*conn).serve+0x6f3 /usr/local/go/src/net/http/server.go:1925
```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reported write path in tsdb/engine/tsm1/wal.go and the task scheduler entry point at task/backend/scheduler/treescheduler.go, then compare it with v1/coordinator/points_writer.go under the reproduction workload. Confirm the source of the recurring timeouts and verify that the bucket, dashboard, alert, and high-rate measurement setup no longer produces recorder or task-executor timeout errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, linux
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100