elastic / elastic/beats

[bug][filebeat] `system.auth` module grok fails for `pam_unix`

Open
#34,247 8 comments 0 reactions 1 assignee Claimed by @andrewkroh View on GitHub
Team:Elastic-Agent Team:Elastic-Agent-Data-Plane
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

Filebeat `system` pipeline fails to grok `pam_unix(sudo:session)` messages in `/var/log/secure`.

https://github.com/elastic/beats/blob/bb64650539673317fffdd231b2c7d06ba2194090/filebeat/module/system/auth/ingest/pipeline.yml#L40

- Version: 8.5.3
- OS: Amazon Linux 2
- Steps to Reproduce:

Enable `system` module with `filebeat modules enable`, and enable the `auth` fileset.

```yaml
# modules.d/system.yml
# Module: system
# Docs: https://www.elastic.co/guide/en/beats/filebeat/8.5/filebeat-module-system.html

- module: system
# Syslog
syslog:
enabled: true

# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
#var.paths:

# Authorization logs
auth:
enabled: true

# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
#var.paths:
```

Then become root from ec2-user with `sudo -i`.

Redacted output:

```json
{
"agent": {
"type": "filebeat",
"version": "8.5.3"
},
"@timestamp": "2023-01-12T18:17:05.112Z",
"ecs": {
"version": "8.0.0"
},
"log": {
"file": {
"path": "/var/log/secure"
},
"offset": 27533
},
"fileset": {
"name": "auth"
},
"message": [
"Jan 12 18:10:48 REDACTED sudo: pam_unix(sudo:session): session opened for user root by ec2-user(uid=0)"
],
"error.message": [
"Provided Grok expressions do not match field value: [{\"@timestamp\":\"2023-01-12T18:17:05.112Z\",\"@metadata\":{\"beat\":\"filebeat\",\"type\":\"_doc\",\"version\":\"8.5.3\",\"pipeline\":\"filebeat-8.5.3-system-auth-pipeline\"},\"fileset\":{\"name\":\"auth\"},\"ecs\":{\"version\":\"8.0.0\"},\"log\":{\"file\":{\"path\":\"/var/log/secure\"},\"offset\":27533},\"service\":{\"type\":\"system\"},\"input\":{\"type\":\"log\"},\"agent\":{\"id\":\"REDACTED\",\"name\":\"REDACTED.compute.internal\",\"type\":\"filebeat\",\"version\":\"8.5.3\",\"ephemeral_id\":\"REDACTED\"},\"message\":\"Jan 12 18:10:48 REDACTED sudo: pam_unix(sudo:session): session opened for user root by ec2-user(uid=0)\",\"event\":{\"module\":\"system\",\"dataset\":\"system.auth\",\"timezone\":\"+00:00\"},\"host\":{\"name\":\"REDACTED.compute.internal\"}}]"
],
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.