elastic / elastic/beats

[Flaky Test] filebeat/input/redis TestInput

Open
#44,247 5 comments 1 reaction 0 assignees View on GitHub
flaky-test Team:Elastic-Agent-Data-Plane
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

## Flaky Test

* **Test Name:** filebeat/input/redis TestInput
* **Link:** [filebeat/input/redis/redis_integration_test.go](https://github.com/elastic/beats/blob/23795862a751911ffa5e4d04c245f278115af516/filebeat/input/redis/redis_integration_test.go#L80).
* **Branch:** main, AndersonQ:44230-fix-panic-duplicated-metrics, AndersonQ:42762-dropped-event-per-input
* **Artifact Link:**
- main: https://buildkite.com/elastic/filebeat/builds/17192/summary/annotations?sid=0196aa0d-64e4-423a-80f4-93947553cac2
- AndersonQ:42762-dropped-event-per-input: https://buildkite.com/elastic/filebeat/builds/17220/summary/annotations?sid=0196ab3e-54de-4a07-86da-8bfd29265b99
- AndersonQ:44230-fix-panic-duplicated-metrics: https://buildkite.com/elastic/filebeat/builds/17217/steps?jid=0196ab40-116d-48b4-8e3f-b9e34c38668b
* **Notes:** Additional details about the test. e.g. theory as to failure cause

### Stack Trace

```
=== Failed

=== FAIL: filebeat/input/redis TestInput (unknown)
logger.go:146: 2025-05-07T15:19:41.794Z DEBUG redis logp/logger.go:201 Run redis input with hosts: [localhost:6380]
coverage: 72.3% of statements
panic: test timed out after 10m0s
running tests:
```

```
2025-05-07 17:11:37 CEST
=== Failed
2025-05-07 17:11:37 CEST
=== FAIL: filebeat/input/redis TestInput (unknown)
2025-05-07 17:11:37 CEST
logger.go:146: 2025-05-07T15:01:37.530Z DEBUG redis logp/logger.go:201 Run redis input with hosts: [localhost:6380]
2025-05-07 17:11:37 CEST
coverage: 72.3% of statements
2025-05-07 17:11:37 CEST
panic: test timed out after 10m0s
2025-05-07 17:11:37 CEST
running tests:
2025-05-07 17:11:37 CEST
TestInput (10m0s)
2025-05-07 17:11:37 CEST
2025-05-07 17:11:37 CEST
goroutine 25 [running]:
2025-05-07 17:11:37 CEST
testing.(*M).startAlarm.func1()
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:2484 +0x394
2025-05-07 17:11:37 CEST
created by time.goFunc
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/time/sleep.go:215 +0x2d
2025-05-07 17:11:37 CEST
2025-05-07 17:11:37 CEST
goroutine 1 [chan receive]:
2025-05-07 17:11:37 CEST
testing.(*T).Run(0xc0001bac40, {0x15b40b1?, 0xc00031fb30?}, 0x1660300)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:1859 +0x431
2025-05-07 17:11:37 CEST
testing.runTests.func1(0xc0001bac40)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:2279 +0x37
2025-05-07 17:11:37 CEST
testing.tRunner(0xc0001bac40, 0xc00031fc70)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:1792 +0xf4
2025-05-07 17:11:37 CEST
testing.runTests(0xc000238e88, {0x1f4d9c0, 0x2, 0x2}, {0x1fc7ec0?, 0x7?, 0x1fc6680?})
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:2277 +0x4b4
2025-05-07 17:11:37 CEST
testing.(*M).Run(0xc0002b35e0)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:2142 +0x64a
2025-05-07 17:11:37 CEST
main.main()
2025-05-07 17:11:37 CEST
_testmain.go:57 +0x9b
2025-05-07 17:11:37 CEST
2025-05-07 17:11:37 CEST
goroutine 7 [sync.WaitGroup.Wait]:
2025-05-07 17:11:37 CEST
sync.runtime_SemacquireWaitGroup(0xc000061d68?)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/runtime/sema.go:110 +0x25
2025-05-07 17:11:37 CEST
sync.(*WaitGroup).Wait(0xc0003568c0?)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/sync/waitgroup.go:118 +0x48
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/harvester.(*Registry).WaitForCompletion(...)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/harvester/registry.go:70
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/harvester.(*Registry).Stop.func1()
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/harvester/registry.go:56 +0x78
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/harvester.(*Registry).Stop(0xc0003568c0)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/harvester/registry.go:66 +0x1a6
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/input/redis.(*Input).Stop(0xc0003404d0)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/input/redis/input.go:124 +0x5a
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/input/redis.TestInput.func3()
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/input/redis/redis_integration_test.go:124 +0x1c
2025-05-07 17:11:37 CEST
testing.(*common).Cleanup.func1()
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:1211 +0x106
2025-05-07 17:11:37 CEST
testing.(*common).runCleanup(0xc0001bae00, 0xc000355b10?)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:1445 +0xea
2025-05-07 17:11:37 CEST
testing.tRunner.func2()
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:1786 +0x25
2025-05-07 17:11:37 CEST
testing.tRunner(0xc0001bae00, 0x1660300)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:1798 +0x11e
2025-05-07 17:11:37 CEST
created by testing.(*T).Run in goroutine 1
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/.asdf/installs/golang/1.24.1/go/src/testing/testing.go:1851 +0x413
2025-05-07 17:11:37 CEST
2025-05-07 17:11:37 CEST
goroutine 15 [chan send]:
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/input/redis.(*eventCaptor).OnEvent(0x17fda07?, {{0x0, 0xedfad6ca5, 0x0}, 0x0, 0xc00038a210, {0x0, 0x0}, 0x0})
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/input/redis/redis_integration_test.go:64 +0x45
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/channel.SubOutlet.func1()
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/channel/util.go:45 +0x110
2025-05-07 17:11:37 CEST
created by github.com/elastic/beats/v7/filebeat/channel.SubOutlet in goroutine 7
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/channel/util.go:43 +0x106
2025-05-07 17:11:37 CEST
2025-05-07 17:11:37 CEST
goroutine 20 [select]:
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/channel.(*subOutlet).OnEvent(0xc0003594a0, {{0x0, 0xedfad6ca5, 0x0}, 0x0, 0xc00038a210, {0x0, 0x0}, 0x0})
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/channel/util.go:83 +0x1d6
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/harvester.(*Forwarder).Send(0x156cb00?, {{0x0, 0xedfad6ca5, 0x0}, 0x0, 0xc00038a210, {0x0, 0x0}, 0x0})
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/harvester/forwarder.go:50 +0xcb
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/input/redis.(*Harvester).Run(0xc000444750)
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/input/redis/harvester.go:196 +0x118e
2025-05-07 17:11:37 CEST
github.com/elastic/beats/v7/filebeat/harvester.(*Registry).Start.func1()
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/harvester/registry.go:96 +0x7a
2025-05-07 17:11:37 CEST
created by github.com/elastic/beats/v7/filebeat/harvester.(*Registry).Start in goroutine 7
2025-05-07 17:11:37 CEST
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1746629844795851813/elastic/filebeat/filebeat/harvester/registry.go:90 +0x1f3
2025-05-07 17:11:37 CEST
2025-05-07 17:11:37 CEST
DONE 1140 tests, 2 skipped, 1 failure in 609.699s
2025-05-07 17:11:48 CEST
Error: failed to execute go: exit status 1
```

Contributor guide

Open the contributing guide

Research direction

Start with TestInput in filebeat/input/redis/redis_integration_test.go, especially the eventCaptor and cleanup around lines 64, 80, and 124. Reproduce the timeout and read the stack trace through input.go, harvester.go, and channel/util.go. Done means the Redis integration test completes reliably without the 10-minute timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, redis
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.