elastic / elastic/beats

[Filebeat] Envoyproxy Module: Support for Envoy Gateway default JSON log format and GeoIP enrichment

Open
#48,618 1 comment 4 reactions 0 assignees View on GitHub
needs_team
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 2h
Merged PRs (30d)
364

Description

**Describe the enhancement:**
Currently, the `envoyproxy` Filebeat module is optimized for standard Envoy Proxy access logs. We are requesting an update to the `envoyproxy/log/ingest` pipelines to natively support **Envoy Gateway** log formats. Specifically:

* **Relaxed Field Requirements:** The `pipeline-entry.yml` currently fails with `pipeline-plaintext: cannot access method/field [charAt] from a null def reference` when the `message` field is absent. The pipeline should be updated to handle JSON logs that do not explicitly contain a `message` key.
* **GeoIP/ECS Mapping for Gateway Schema:** The module should be enhanced to map Envoy Gateway's JSON fields (such as `x-forwarded-for` and `downstream_remote_address`) to standard ECS fields like `source.ip` and `client.ip` to enable GeoIP enrichment.
* **Out-of-the-box Dashboards:** Provide map-based visualizations for Envoy Gateway traffic, similar to the functionality currently available in the NGINX and Traefik modules.

**Describe a specific use case for the enhancement or feature:**
As users migrate from NGINX Ingress to **Envoy Gateway** (often due to the upcoming NGINX Ingress Controller retirement in 2026), they require the same level of observability.

The default Envoy Gateway access log format (documented by the vendor here: [https://gateway.envoyproxy.io/docs/tasks/observability/proxy-accesslog/#default-access-log](https://gateway.envoyproxy.io/docs/tasks/observability/proxy-accesslog/#default-access-log)) outputs JSON prefixed with a string (e.g., `envoy {...}`) but does not include a `message` field. Because the current ingest pipeline logic at `x-pack/filebeat/module/envoyproxy/log/ingest/pipeline-entry.yml` relies on `ctx.message.charAt(0)`, the module fails to parse these logs entirely, preventing any GeoIP enrichment or dashboarding.

**Mockup Data (Envoy Gateway Default JSON):**

```json
envoy {":authority":"api.example.app","bytes_received":0,"bytes_sent":35222,"downstream_local_address":"10.0.0.50:10443","downstream_remote_address":"203.0.113.1:0","duration":2,"method":"GET","protocol":"HTTP/1.1","response_code":200,"start_time":"2026-01-27T10:46:53.463Z","upstream_cluster":"httproute/default/api-rule/0","x-envoy-origin-path":"/","x-forwarded-for":"203.0.113.1, 10.0.0.1","x-request-id":"9f9aab8a-fe0d-4127-9588-ca91f9bbf4e6"}

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.