Filebeat and container metadata with containerd+nerdctl (no k8s)
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 15m
- Merged PRs (30d)
- 385
Description
Hi!
We use containerd+nerdctl and compose schema. We have not so many apps and resources to use k8s. Now we want to ship logs to remote EFK so we can analyse them. Trouble is that we could not get container metadata with filebeat. `add_docker_metadata` dont work with containerd socket obviously, nerdctl place containers dirs only by there long id.
Is there any way to config log collecting with metadata from containerd?
In containers config we got this
```
{
...
"annotations": {
...
"com.docker.compose.project": "my-project",
"com.docker.compose.service": "my-app",
"nerdctl/name": "container-name",
...
}
....
}
```
Is it possible for filebeat to get it?
I think it could be useful for k8s users whenever they want to use filebeat for log collecting k8s and non-k8s machines and deploy filebeat to OS not in k8s.
Contributor guide
Research direction
The issue names Filebeat's add_docker_metadata input, the containerd socket, and container annotations; start by tracing the metadata processor's supported runtime inputs. Confirm whether compose and nerdctl annotations can be exposed for non-Kubernetes containerd logs, and define completion as those fields appearing in shipped events with a documented configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100