[Auditbeat] `test_file_modified__ebpf`, `test_non_recursive__ebpf` and `test_recursive__ebpf`: Timeout waiting for condition 'log_contains'. Waited 30 seconds
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 15m
- Merged PRs (30d)
- 385
Description
## Flaky Test
* **Test Name:** `test_file_modified__ebpf`, `test_non_recursive__ebpf` and `test_recursive__ebpf`
* **Link:** https://github.com/elastic/beats/blob/main/auditbeat/tests/system/test_file_integrity.py
* **Branch:** `main`
* **Artifact Link:**
- https://buildkite.com/elastic/auditbeat/builds/21617/steps/canvas?jid=01996255-703f-4c38-b4e6-f9ed841ce5e9#01996255-703f-4c38-b4e6-f9ed841ce5e9/7-2279
- https://buildkite.com/elastic/auditbeat/builds/21614/steps/canvas?jid=0199622a-4990-45da-9da5-0b5f83bb0984#0199622a-4990-45da-9da5-0b5f83bb0984/7-2276
- https://buildkite.com/elastic/auditbeat/builds/21604/steps/canvas?jid=01996162-c27b-428c-b256-516d9d9de2c3#01996162-c27b-428c-b256-516d9d9de2c3/7-2278
### Stack Trace
```
2025-09-19 06:06:05 EDT
=================================================================== short test summary info ====================================================================
2025-09-19 06:06:05 EDT
FAILED tests/system/test_file_integrity.py::Test::test_file_modified__ebpf - beat.beat.WaitTimeoutError: Timeout waiting for condition 'log_contains'. Waited 30 seconds:
2025-09-19 06:06:05 EDT
FAILED tests/system/test_file_integrity.py::Test::test_non_recursive__ebpf - beat.beat.WaitTimeoutError: Timeout waiting for condition 'log_contains'. Waited 30 seconds:
2025-09-19 06:06:05 EDT
FAILED tests/system/test_file_integrity.py::Test::test_recursive__ebpf - beat.beat.WaitTimeoutError: Timeout waiting for condition 'log_contains'. Waited 30 seconds:
2025-09-19 06:06:05 EDT
===================================================== 3 failed, 17 passed, 2 warnings in 256.52s (0:04:16) =====================================================
2025-09-19 06:06:06 EDT
```
It seems all tests related to eBPF are failing. I ran some of the tests locally and got this error:
```json
{
"log.level": "error",
"@timestamp": "2025-09-19T11:53:51.152-0400",
"log.logger": "file_integrity",
"log.origin": {
"function": "github.com/elastic/beats/v7/auditbeat/module/file_integrity.(*MetricSet).init",
"file.name": "file_integrity/metricset.go",
"file.line": 195
},
"message": "Failed to initialize",
"service.name": "auditbeat",
"error": {
"message": "failed to start event producer: init ebpf loader: load bpf: rlimit remove memlock: failed to set memlock rlimit: operation not permitted"
},
"ecs.version": "1.6.0"
}
{
"log.level": "error",
"@timestamp": "2025-09-19T15:44:49.936Z",
"log.logger": "file_integrity",
"log.origin": {
"function": "github.com/elastic/beats/v7/auditbeat/module/file_integrity.(*MetricSet).init",
"file.name": "file_integrity/metricset.go",
"file.line": 195
},
"message": "Failed to initialize",
"service.name": "auditbeat",
"error": {
"message": "failed to start event producer: init ebpf loader: load bpf: verifier error: field FentryTtyWrite: program fentry__tty_write: load program: bad CO-RE relocation: invalid func unknown#195896080 (885 line(s) omitted)"
},
"ecs.version": "1.6.0"
}
{
"log.level": "error",
"@timestamp": "2025-09-19T15:45:49.123Z",
"log.logger": "file_integrity",
"log.origin": {
"function": "github.com/elastic/beats/v7/auditbeat/module/file_integrity.(*MetricSet).init",
"file.name": "file_integrity/metricset.go",
"file.line": 195
},
"message": "Failed to initialize",
"service.name": "auditbeat",
"error": {
"message": "failed to start event producer: init ebpf loader: load bpf: verifier error: field FentryTtyWrite: program fentry__tty_write: load program: bad CO-RE relocation: invalid func unknown#195896080 (885 line(s) omitted)"
},
"ecs.version": "1.6.0"
}
```
Contributor guide
Research direction
Start with auditbeat/tests/system/test_file_integrity.py and reproduce the three named eBPF tests. Then inspect file_integrity/metricset.go around MetricSet.init and compare the reported memlock and CO-RE verifier errors. Done means the eBPF tests no longer time out and the related tests pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, linux
- Domain
- operating-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100