elastic / elastic/beats

Auditbeat: file_integrity module tests failure

Open
#40,396 3 comments 0 reactions 0 assignees View on GitHub
Auditbeat flaky-test Team:Security-Service Integrations
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 2h
Merged PRs (30d)
364

Description

## Flaky Test

### auditbeat/module/file_integrity/fileinfo_test.go
* **Test Name:** TestNewMetadata
**Link:** https://github.com/elastic/beats/blob/49f3efeada111d683754c2615b6af0e847a4c998/auditbeat/module/file_integrity/fileinfo_test.go#L32-L98

* **Test Name:** TestSetUIDSetGIDBits
**Link:** https://github.com/elastic/beats/blob/49f3efeada111d683754c2615b6af0e847a4c998/auditbeat/module/file_integrity/fileinfo_test.go#L100-L156

### auditbeat/module/file_integrity/metricset_test.go
* **Test Name:** TestActions
**Link:** https://github.com/elastic/beats/blob/49f3efeada111d683754c2615b6af0e847a4c998/auditbeat/module/file_integrity/metricset_test.go#L64-L159

* **Test Name:** TestExcludedFiles
**Link:** https://github.com/elastic/beats/blob/49f3efeada111d683754c2615b6af0e847a4c998/auditbeat/module/file_integrity/metricset_test.go#L161-L211

* **Test Name:** TestIncludedExcludedFiles
**Link:** https://github.com/elastic/beats/blob/49f3efeada111d683754c2615b6af0e847a4c998/auditbeat/module/file_integrity/metricset_test.go#L213-L271

* **Branch:**
-- https://github.com/elastic/beats/pull/40387
-- https://github.com/elastic/beats/pull/40398

**NB!** tests should be enabled back, since those were muted while migrating from Jenkins to Buildkite. Please check changed files in mentioned PRs:
-- auditbeat/module/file_integrity/fileinfo_test.go
-- auditbeat/module/file_integrity/metricset_test.go

* **Artifact Link:**
macOS arm64: [auditbeat_build_6789_mac-auditbeat-macos-arm64-unit-tests.log](https://github.com/user-attachments/files/16430460/auditbeat_build_6789_mac-auditbeat-macos-arm64-unit-tests.log)
macOS x86_64: [auditbeat_build_6789_mac-auditbeat-macos-x86-64-unit-tests.log](https://github.com/user-attachments/files/16430466/auditbeat_build_6789_mac-auditbeat-macos-x86-64-unit-tests.log)
win 2016 (other win failures are similar): [auditbeat_build_6789_windows-auditbeat-win-2016-unit-tests.log](https://github.com/user-attachments/files/16430762/auditbeat_build_6789_windows-auditbeat-win-2016-unit-tests.log)

* **Notes:** Additional details about the test. e.g. theory as to failure cause
1. `metricset_test.go` failes on Windows due to shortened TMP, additional details: https://github.com/elastic/ingest-dev/issues/3016

2. Buildkite builds:
-- macOS x86_64: https://buildkite.com/elastic/auditbeat/builds/6789#019102f4-24b0-478d-9549-970404e44521
-- macOS arm64: https://buildkite.com/elastic/auditbeat/builds/6789#019102f4-24b2-43b3-ac36-04b9cdcdba09
-- Win 2016: https://buildkite.com/elastic/auditbeat/builds/6789#019102f4-24a1-4cdb-9df0-10fb8a5c0151
-- Win 2022: https://buildkite.com/elastic/auditbeat/builds/6789#019102f4-24a3-417f-9bd1-a0895f3d485a
-- Win 2019: https://buildkite.com/elastic/auditbeat/builds/6789#019102f4-24b5-4e2b-8e80-cd65acbf9bea
-- Win 10: https://buildkite.com/elastic/auditbeat/builds/6789#019102f4-24b7-4b04-8fd3-5e3e257cddf1
-- Win 11: https://buildkite.com/elastic/auditbeat/builds/6789#019102f4-24b9-48ab-916b-f3e42892e0eb
-- macOS x86_64 on 7.17 branch: https://buildkite.com/elastic/auditbeat/builds/6826#019107ea-0217-4614-bfd5-e8941bac565f/1893-1943

3. Tests failure per OS:

| Beat | Test file | Test | OS |
|:----------|:----------------------------------------------------|:--------------------------|:---------------------------------------------------|
| auditbeat | `auditbeat/module/file_integrity/fileinfo_test.go` | TestNewMetadata | macOS x86_64
macOS arm64 |
| auditbeat | `auditbeat/module/file_integrity/fileinfo_test.go` | TestSetUIDSetGIDBits | macOS x86_64
macOS arm64 |
| auditbeat | `auditbeat/module/file_integrity/metricset_test.go` | TestActions | macOS arm64
Windows (2016, 2019, 2022, 10, 11) |
| auditbeat | `auditbeat/module/file_integrity/metricset_test.go` | TestExcludedFiles | macOS arm64
Windows (2016, 2019, 2022, 10, 11) |
| auditbeat | `auditbeat/module/file_integrity/metricset_test.go` | TestIncludedExcludedFiles | macOS arm64
Windows (2016, 2019, 2022, 10, 11) |

### Stack Trace

```
=== FAIL: auditbeat/module/file_integrity TestNewMetadata (0.02s)
--
  | fileinfo_test.go:86:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/fileinfo_test.go:86
  | Error: Not equal:
  | expected: "20"
  | actual : "0"
  |  
  | Diff:
  | --- Expected
  | +++ Actual
  | @@ -1 +1 @@
  | -20
  | +0
  | Test: TestNewMetadata
  | fileinfo_test.go:88:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/fileinfo_test.go:88
  | Error: Not equal:
  | expected: "staff"
  | actual : "wheel"
  |  
  | Diff:
  | --- Expected
  | +++ Actual
  | @@ -1 +1 @@
  | -staff
  | +wheel
  | Test: TestNewMetadata
  |  
  | === FAIL: auditbeat/module/file_integrity TestSetUIDSetGIDBits (0.02s)
  | fileinfo_test.go:154:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/fileinfo_test.go:154
  | Error: Not equal:
  | expected: true
  | actual : false
  | Test: TestSetUIDSetGIDBits
  |  
  | === FAIL: auditbeat/module/file_integrity TestActions (10.22s)
  | metricset_test.go:128:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/metricset_test.go:128
  | Error: "[{{} {} {"event":{"action":["initial_scan"],"category":["file"],"kind":"event","type":["info"]},"file":{"ctime":"2024-07-30T09:33:55.318592553Z","gid":"0","group":"wheel","inode":"520903","mode":"0755","mtime":"2024-07-30T09:33:55.318592553Z","owner":"admin","path":"/private/tmp/TestActions4021485366/001","type":"dir","uid":"501"}} 2024-07-30 09:33:55.347081 +0000 UTC 148.042µs 0s false} {{} {} {"event":{"action":["initial_scan"],"category":["file"],"kind":"event","type":["info"]},"file":{"ctime":"2024-07-30T09:33:55.31849897Z","extension":"txt","gid":"0","group":"wheel","hash":{"sha1":"2aae6c35c94fcfb415dbe95f408b9ce91ee846ed"},"inode":"520907","mode":"0600","mtime":"2024-07-30T09:33:55.31849897Z","owner":"admin","path":"/private/tmp/TestActions4021485366/001/created.txt","size":11,"type":"file","uid":"501"}} 2024-07-30 09:33:55.347173 +0000 UTC 102.875µs 0s false} {{} {} {"event":{"action":["initial_scan"],"category":["file"],"kind":"event","type":["info"]},"file":{"ctime":"2024-07-30T09:33:55.31863272Z","extension":"txt","gid":"0","group":"wheel","hash":{"sha1":"2aae6c35c94fcfb415dbe95f408b9ce91ee846ed"},"inode":"520908","mode":"0600","mtime":"2024-07-30T09:33:55.31863272Z","owner":"admin","path":"/private/tmp/TestActions4021485366/001/updated.txt","size":11,"type":"file","uid":"501"}} 2024-07-30 09:33:55.347275 +0000 UTC 76.458µs 0s false} {{} {} {"event":{"action":["initial_scan"],"category":["file"],"kind":"event","type":["info"]},"file":{"ctime":"2024-07-30T09:33:55.182588173Z","gid":"0","group":"wheel","inode":"520904","mode":"0755","mtime":"2024-07-30T09:33:55.182588173Z","owner":"admin","path":"/private/tmp/TestActions4021485366/002","type":"dir","uid":"501"}} 2024-07-30 09:33:55.373103 +0000 UTC 101.084µs 0s false}]" should have 5 item(s), but has 4
  | Test: TestActions
  | metricset_test.go:155:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/metricset_test.go:155
  | Error: unexpected path
  | Test: TestActions
  | Messages: path /private/tmp/TestActions4021485366/001
  | metricset_test.go:155:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/metricset_test.go:155
  | Error: unexpected path
  | Test: TestActions
  | Messages: path /private/tmp/TestActions4021485366/001/created.txt
  | metricset_test.go:155:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/metricset_test.go:155
  | Error: unexpected path
  | Test: TestActions
  | Messages: path /private/tmp/TestActions4021485366/001/updated.txt
  | metricset_test.go:155:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/metricset_test.go:155
  | Error: unexpected path
  | Test: TestActions
  | Messages: path /private/tmp/TestActions4021485366/002
  |  
  | === FAIL: auditbeat/module/file_integrity TestExcludedFiles (0.17s)
  | metricset_test.go:208:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/metricset_test.go:208
  | Error: Should be true
  | Test: TestExcludedFiles
  | metricset_test.go:208:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/metricset_test.go:208
  | Error: Should be true
  | Test: TestExcludedFiles
  | metricset_test.go:208:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/metricset_test.go:208
  | Error: Should be true
  | Test: TestExcludedFiles
  |  
  | === FAIL: auditbeat/module/file_integrity TestIncludedExcludedFiles (0.16s)
  | metricset_test.go:270:
  | Error Trace: /Users/admin/builds/bk-agent-prod-orka-1722331758191225599/elastic/auditbeat/auditbeat/module/file_integrity/metricset_test.go:270
  | Error: Not equal:
  | expected: map[string]bool{"/tmp/TestIncludedExcludedFiles2240735922/001":true, "/tmp/TestIncludedExcludedFiles2240735922/001/.ssh":true, "/tmp/TestIncludedExcludedFiles2240735922/001/.ssh/known_hosts":true}
  | actual : map[string]bool{"/private/tmp/TestIncludedExcludedFiles2240735922/001":true, "/private/tmp/TestIncludedExcludedFiles2240735922/001/.ssh":true, "/private/tmp/TestIncludedExcludedFiles2240735922/001/.ssh/known_hosts":true}
  |  
  | Diff:
  | --- Expected
  | +++ Actual
  | @@ -1,5 +1,5 @@
  | (map[string]bool) (len=3) {
  | - (string) (len=44) "/tmp/TestIncludedExcludedFiles2240735922/001": (bool) true,
  | - (string) (len=49) "/tmp/TestIncludedExcludedFiles2240735922/001/.ssh": (bool) true,
  | - (string) (len=61) "/tmp/TestIncludedExcludedFiles2240735922/001/.ssh/known_hosts": (bool) true
  | + (string) (len=52) "/private/tmp/TestIncludedExcludedFiles2240735922/001": (bool) true,
  | + (string) (len=57) "/private/tmp/TestIncludedExcludedFiles2240735922/001/.ssh": (bool) true,
  | + (string) (len=69) "/private/tmp/TestIncludedExcludedFiles2240735922/001/.ssh/known_hosts": (bool) true
  | }
  | Test: TestIncludedExcludedFiles
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.