[Flaky Test] libbeat/api: Log in goroutine after TestHTTP has completed:
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 15m
- Merged PRs (30d)
- 385
Description
## Flaky Test
* **Test Name:** libbeat/api
* **Link:** https://github.com/elastic/beats/tree/main/libbeat/api
* **Branch:** main, AndersonQ:42762-dropped-event-per-input
* **Artifact Link:** [beats-libbeat_build_16509_ubuntu-libbeat-go-integration-tests.log.txt](https://github.com/user-attachments/files/20705707/beats-libbeat_build_16509_ubuntu-libbeat-go-integration-tests.log.txt)
* **Notes:** Additional details about the test. e.g. theory as to failure cause
It's a known Go issue that can be avoided: https://github.com/golang/go/issues/30389
It's easy to reproduce it ~(on my machine)~ with
```
go test -count 20000 ./libbeat/api
panic: Log in goroutine after TestHTTP has completed: 2025-06-12T11:30:47.151+0200 INFO api logp/logger.go:206 Stats endpoint (127.0.0.1:33591) finished: accept tcp 127.0.0.1:33591: use of closed network connection
goroutine 475761 [running]:
testing.(*common).logDepth(0xc063f40fc0, {0xc0632726e0, 0xa0}, 0x3)
/home/ainsoph/devel/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.3.linux-amd64/src/testing/testing.go:1064 +0x4b4
testing.(*common).log(...)
/home/ainsoph/devel/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.3.linux-amd64/src/testing/testing.go:1046
testing.(*common).Logf(0xc063f40fc0, {0x8c5036?, 0x400?}, {0xc04cab6240?, 0x795811?, 0x3?})
/home/ainsoph/devel/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.3.linux-amd64/src/testing/testing.go:1097 +0x54
go.uber.org/zap/zaptest.TestingWriter.Write({{0x7f99c9aad2c8?, 0xc063f40fc0?}, 0xc0?}, {0xc04ca6c800?, 0xa1, 0xc04ca6c800?})
/home/ainsoph/devel/go/pkg/mod/go.uber.org/zap@v1.27.0/zaptest/logger.go:146 +0xdb
go.uber.org/zap/zapcore.(*ioCore).Write(0xc04ce250e0, {0x0, {0xc20ac571c906f4ba, 0x6d992bf72, 0xc52e00}, {0x8c517e, 0x3}, {0xc04d09e8c0, 0x67}, {0x1, ...}, ...}, ...)
/home/ainsoph/devel/go/pkg/mod/go.uber.org/zap@v1.27.0/zapcore/core.go:99 +0xb5
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc03c291d40, {0x0, 0x0, 0x0})
/home/ainsoph/devel/go/pkg/mod/go.uber.org/zap@v1.27.0/zapcore/entry.go:253 +0x119
go.uber.org/zap.(*SugaredLogger).log(0xc0632689b0, 0x0, {0x8d2d99?, 0x20?}, {0xc062145f70?, 0x2?, 0x2?}, {0x0, 0x0, 0x0})
/home/ainsoph/devel/go/pkg/mod/go.uber.org/zap@v1.27.0/sugar.go:355 +0xec
go.uber.org/zap.(*SugaredLogger).Infof(...)
/home/ainsoph/devel/go/pkg/mod/go.uber.org/zap@v1.27.0/sugar.go:204
github.com/elastic/elastic-agent-libs/logp.(*Logger).Infof(...)
/home/ainsoph/devel/go/pkg/mod/github.com/elastic/elastic-agent-libs@v0.19.6/logp/logger.go:206
github.com/elastic/beats/v7/libbeat/api.(*Server).Start.func1({0x96a670, 0xc06383cec0})
/home/ainsoph/devel/github.com/elastic/beats/libbeat/api/server.go:69 +0x20c
created by github.com/elastic/beats/v7/libbeat/api.(*Server).Start in goroutine 475769
/home/ainsoph/devel/github.com/elastic/beats/libbeat/api/server.go:66 +0x110
FAIL github.com/elastic/beats/v7/libbeat/api 30.118s
FAIL
```
### Stack Trace
```
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.3/go/src/testing/testing.go:1064 +0x4b4
2025-06-11 18:23:30 CEST
testing.(*common).log(...)
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.3/go/src/testing/testing.go:1046
2025-06-11 18:23:30 CEST
testing.(*common).Logf(0xc0000ff880, {0x8e8176?, 0x400?}, {0xc00003d8b0?, 0x7b2f43?, 0x3?})
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.3/go/src/testing/testing.go:1097 +0x54
2025-06-11 18:23:30 CEST
go.uber.org/zap/zaptest.TestingWriter.Write({{0x71ef9643e310?, 0xc0000ff880?}, 0x80?}, {0xc0001fa000?, 0x9d, 0xc0001fa000?})
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.3/packages/pkg/mod/go.uber.org/zap@v1.27.0/zaptest/logger.go:146 +0xdb
2025-06-11 18:23:30 CEST
go.uber.org/zap/zapcore.(*ioCore).Write(0xc00019ecc0, {0x0, {0xc20a88a792730549, 0x73dcc0, 0xc85f60}, {0x8e82c3, 0x3}, {0xc000035650, 0x67}, {0x1, ...}, ...}, ...)
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.3/packages/pkg/mod/go.uber.org/zap@v1.27.0/zapcore/core.go:99 +0xb5
2025-06-11 18:23:30 CEST
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0001a01a0, {0x0, 0x0, 0x0})
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.3/packages/pkg/mod/go.uber.org/zap@v1.27.0/zapcore/entry.go:253 +0x119
2025-06-11 18:23:30 CEST
go.uber.org/zap.(*SugaredLogger).log(0xc00007c298, 0x0, {0x8f64ca?, 0x20?}, {0xc000200f70?, 0x2?, 0x2?}, {0x0, 0x0, 0x0})
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.3/packages/pkg/mod/go.uber.org/zap@v1.27.0/sugar.go:355 +0xec
2025-06-11 18:23:30 CEST
go.uber.org/zap.(*SugaredLogger).Infof(...)
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.3/packages/pkg/mod/go.uber.org/zap@v1.27.0/sugar.go:204
2025-06-11 18:23:30 CEST
github.com/elastic/elastic-agent-libs/logp.(*Logger).Infof(...)
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.3/packages/pkg/mod/github.com/elastic/elastic-agent-libs@v0.19.6/logp/logger.go:206
2025-06-11 18:23:30 CEST
github.com/elastic/beats/v7/libbeat/api.(*Server).Start.func1({0x991510, 0xc0000a56c0})
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1749657779978475881/elastic/beats-libbeat/libbeat/api/server.go:69 +0x21e
2025-06-11 18:23:30 CEST
created by github.com/elastic/beats/v7/libbeat/api.(*Server).Start in goroutine 37
2025-06-11 18:23:30 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1749657779978475881/elastic/beats-libbeat/libbeat/api/server.go:66 +0x14c
2025-06-11 18:23:30 CEST
FAIL github.com/elastic/beats/v7/libbeat/api 0.018s
```
Contributor guide
Research direction
Start by running `go test -count 20000 ./libbeat/api` and read `libbeat/api/server.go` around the goroutine at lines 66-69, along with the `TestHTTP` setup and teardown. Trace the server shutdown path and verify the test cannot complete while that goroutine can still log; done means the repeated package test finishes without the post-test logging panic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100