GA support for reading from journald
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 15m
- Merged PRs (30d)
- 385
Description
- Relates https://github.com/elastic/elastic-agent/issues/3650
As of Debian 12 system logs are exclusively available via journald by default. Today we support reading journald logs via the Filebeat [journald input](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-journald.html), which is still in technical preview and has several major bugs filed against it. See https://github.com/elastic/beats/issues?q=is%3Aissue+is%3Aopen+journald notably:
- [x] https://github.com/elastic/beats/issues/34077
- [x] https://github.com/elastic/beats/issues/32782
- [x] https://github.com/elastic/beats/issues/30398
We need to provide a GA way to read journald logs. There are two paths to this:
1. Fix the major issues in the journald input and GA it as is. All integrations that previously read syslog files by default will need a conditional to specify that journald should be used instead of one of the log files on Linux (see [example](https://github.com/elastic/integrations/blob/9dcd6eac196eed413eff898c1ecbc46fe1d745e1/packages/iptables/data_stream/log/agent/stream/journald.yml.hbs#L1). Possibly this conditional will need to be on the Linux distribution and not just Linux as a platform.
2. ~Fold the existing journald functionality into filestream, so that there is only one way to read log files and all existing uses of filestream to read system logs continue to work with no or minimal modification. In the ideal case we detect we are reading journald logs based on a .journal extension or well known file paths, but we may need a configuration flag for this. If we do end up with a configuration flag we could consider implementing journald support as a type of parser https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html#_parsers~
Edit:
Option 1 is the path forward, we'll keep the separate journald input.
To close this issue we'll need to:
```[tasklist]
### Must have
- [ ] https://github.com/elastic/integrations/issues/9690
- [x] https://github.com/elastic/beats/issues/34077
- [x] https://github.com/elastic/beats/issues/30398
- [ ] https://github.com/elastic/beats/issues/39352
- [ ] https://github.com/elastic/beats/issues/41083
- [ ] https://github.com/elastic/beats/issues/41353
- [ ] https://github.com/elastic/beats/issues/41378
- [ ] https://github.com/elastic/integrations/issues/10797
- [ ] https://github.com/elastic/beats/issues/42208
- [ ] https://github.com/elastic/beats/issues/41278
- [ ] https://github.com/elastic/integrations/issues/13019
- [ ] https://github.com/elastic/beats/issues/44040
```
```[tasklist]
### Nice to have
- [x] https://github.com/elastic/beats/issues/32782
- [ ] https://github.com/elastic/beats/issues/37877
- [ ] https://github.com/elastic/beats/issues/41331
- [ ] https://github.com/elastic/beats/issues/37876
- [ ] https://github.com/elastic/beats/issues/39791
- [ ] https://github.com/elastic/beats/issues/40185
- [ ] https://github.com/elastic/beats/issues/40478
- [ ] https://github.com/elastic/beats/issues/40479
- [ ] https://github.com/elastic/beats/issues/40526
- [ ] https://github.com/elastic/integrations/issues/10998
```
Contributor guide
Assessment
This issue has not been assessed yet.