elastic / elastic/apm-agent-python
APM agent fails platform detection on cgroupv2 based systems
- Lenguaje dominante
- Python
- Estrellas
- 431
- Forks
- 239
- Merge medio
- 5 d 10 h
- PR fusionados (30 d)
- 7
Descripción
Hello!
Widespread cgroupv2 adoption is around the corner, as many popular distributions already come with cgroupv2 enabled by default.
The problem currently is with the fact, that the APM agent cannot detect that it is running on cgroupv2 based system, as it tries to parse the cgroup file which is fairly empty on a cgroupv2 enabled system. This causes the agent to specify "linux" as it's platform, omit container/pod ID's and cause multiple integrations to fail inside the Elastic ecosystem.
The current cgroup implementation uses the `/proc/self/cgroup` file to obtain relevant container and pod IDs. This file is empty on cgroupv2 based systems, and even in so called cgroupv1 systems, it's an undocumented "feature".
Currently there is no agreed upon method for obtaining this information from inside the container, and it's still a standing issue for the open container spec developers: https://github.com/opencontainers/runtime-spec/issues/1105
My proposal is to use the workaround similar to here:
- https://community.toradex.com/t/python-nullresource-error-when-running-torizoncore-builder-build/15240/4
- https://github.com/open-telemetry/opentelemetry-js-contrib/pull/1181/files
The `/proc/self/mountinfo` file still contains references to necessary information (pod and container uid).
We could use this until at least the standard is agreed upon, and could be switched out.
An example of the relevant information from my latest stable Flatcar Linux system, with Containerd, Kubernetes 1.24.7 and cgroupv2 enabled:
```
5051 5044 259:8 /lib/kubelet/pods/68ee930a-2bd8-447b-8deb-426add7a2d09/etc-hosts /etc/hosts rw,relatime - xfs /dev/nvme0n1p3 rw,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota
5052 5046 259:8 /lib/kubelet/pods/68ee930a-2bd8-447b-8deb-426add7a2d09/containers//2631d4a6 /dev/termination-log rw,relatime - xfs /dev/nvme0n1p3 rw,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota
5053 5044 259:8 /lib/containerd/io.containerd.grpc.v1.cri/sandboxes/199dafcfc5712cd1e9e49e94642e7df6cdf63356bbc3601e9115f26fd0d096e1/hostname /etc/hostname rw,relatime - xfs /dev/nvme0n1p3 rw,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota
5054 5044 259:8 /lib/containerd/io.containerd.grpc.v1.cri/sandboxes/199dafcfc5712cd1e9e49e94642e7df6cdf63356bbc3601e9115f26fd0d096e1/resolv.conf /etc/resolv.conf rw,relatime - xfs /dev/nvme0n1p3 rw,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota
```
The problem which prevents me from working on this issue is that I do not know which formats these lines can take on different systems.
What are your ideas, would this method work?
Guía de contribución
Línea de trabajo
Comienza rastreando el análisis existente de /proc/self/cgroup del agente y compáralo con los ejemplos de /proc/self/mountinfo y el PR de OpenTelemetry enlazado. Confirma los formatos de línea compatibles en distintos entornos antes de definir las pruebas y los criterios de aceptación: la detección de cgroupv2 debe identificar la plataforma y los ID del contenedor o del pod sin afectar al comportamiento de cgroupv1.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- kubernetes, linux, python
- Área
- infrastructure, observability, operating-systems
- Tipo de issue
- Error
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 28/100