cloudnative-pg / cloudnative-pg/cloudnative-pg
[Feature]: Support Otel Log streaming w/o daemonset
- Dominant language
- Go
- Stars
- 9.3k
- Forks
- 759
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 44
Description
### Is there an existing issue already for this feature request/idea?
- [x] I have searched for an existing issue, and could not find anything. I believe this is a new feature request to be evaluated.
### What problem is this feature going to solve? Why should it be added?
Currently, collecting logs from CloudNativePG (operator, instance manager, and PostgreSQL) requires an external log collection infrastructure — either a DaemonSet (e.g. Fluent
Bit, Fluentd) reading container stdout from the node filesystem, or a sidecar tailing log files.
This is problematic because:
1. CNPG's architecture makes external log capture difficult. The operator generates operand pod specs and injects its own instance manager binary via an init container. The
instance manager acts as a custom process manager — PostgreSQL runs as a child process and writes to a CSV FIFO (not stdout). The instance manager reads, parses, and re-emits
these logs through Go's logr interface. There is no shell entrypoint to hook into for stdout redirection.
2. DaemonSets violate Kubernetes best practices for workload portability. The [Red Hat Best Practices for Kubernetes](https://redhat-best-practices-for-k8s.github.io/guide/#k8s-best-practices-no-naked-pods) recommend that applications should not depend on node-level infrastructure or assume DaemonSets are present. Log collection that relies on a
DaemonSet tailing node-local files creates a hard dependency on cluster-wide infrastructure outside the application's control. This makes CNPG non-portable to environments where
DaemonSets are restricted (multi-tenant clusters, managed Kubernetes offerings with locked-down node access, edge deployments, or clusters with strict pod security policies).
3. Sidecar log tailers can't work. CNPG doesn't write to a shared file volume — logs flow through an in-process Go logger to stdout. There is no file for a sidecar to tail. The
LogShipper pattern (Fluent Bit sidecar reading from /logs/) is incompatible because CNPG manages its own pod specs and uses a different log format.
Adding native OpenTelemetry log export solves all of these: logs push directly from the process to an OTel Collector via OTLP gRPC, with full structure preserved, no DaemonSet
dependency, no sidecar, and no modification to the container image or entrypoint. The application becomes self-contained for observability — portable across any Kubernetes
environment regardless of what node-level infrastructure is available.
### Describe the solution you'd like
When configured with an OpenTelemetry Collector endpoint, CNPG natively pushes all its logs — from the operator, the instance manager, and PostgreSQL — directly to the collector via OTLP gRPC. Logs
continue to appear on stdout as they do today; the OTel export is additive.
This covers all three log sources without requiring any external infrastructure (no DaemonSet, no sidecar, no file volume). PostgreSQL log fields (severity, SQL state, query text, session ID, etc.) are
preserved as structured attributes in the exported log records rather than being flattened into a JSON string.
The feature is opt-in via a single environment variable or CLI flag, and has no effect on existing behavior when not configured.
### Describe alternatives you've considered
1. DaemonSet log collector (Fluent Bit / Fluentd): Captures stdout from node-local files. Works but requires cluster-wide infrastructure, adds per-node resource overhead, and is not viable in environments that restrict DaemonSets.
2. LogShipper sidecar (Fluent Bit reading from shared file volume): Incompatible with CNPG. The operator controls pod specs and uses a custom process manager — there is no shared log file volume to tail.
3. Stdout redirect / wrapper script: Not possible. The operand container entrypoint is the instance manager binary injected from the operator image via init container. There is no shell, no
docker-entrypoint.sh, and no interception point between the Go process and stdout.
4. OTel Collector sidecar with filelog receiver: Would require shareProcessNamespace: true to read the main container's stdout from /proc/1/fd/1, or writing logs to a shared volume — neither of which CNPG
supports. Still involves re-parsing JSON that was already structured in-process.
5. Standalone Deployment streaming logs via Kubernetes API: A separate deployment that calls the K8s log API (kubectl logs --follow) for each CNPG pod. Functional but adds API server load and introduces
latency.
### Additional context
The extension would be to the cloudnative-pg/machinery repo where the [OTel Logs API](https://opentelemetry.io/docs/specs/otel/logs/api/) would be extended.
The functionality could be provided as a experimental feature like: [traefik](https://doc.traefik.io/traefik/v3.4/observability/logs/#opentelemetry)
### Backport?
No
### Are you willing to actively contribute to this feature?
Yes
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Contributor guide
Assessment
This issue has not been assessed yet.