envoyproxy / envoyproxy/gateway
Handling OTel traceparent header from external traffic
- Dominant language
- Go
- Stars
- 3k
- Forks
- 864
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 140
Description
*Description*:
When OpenTelemetry tracing is [enabled in the proxy](https://gateway.envoyproxy.io/latest/tasks/observability/proxy-trace/), the `traceparent` header can be used to override trace IDs and disable tracing. This is probably not suitable for an edge facing proxy, where requests come in from outside of the network.
**Example**
If you install the example manifest from the Envoy Gateway docs, you can disable tracing for an individual request by sending a request like this:
```
curl --verbose --header "Host: www.example.com" http://localhost:8888/get -H 'traceparent: 00-66666666666666666666666666666666-6666666666666666-00'
```
The `00` at the end is a trace flag that disables tracing.
The trace context spec here [says this](https://www.w3.org/TR/trace-context/#mutating-the-traceparent-field) about mutating the `traceparent` field for services that are designed as a front gate into secure networks:
> Restart trace: All properties (trace-id, parent-id, trace-flags) are regenerated. This mutation is used in services that are defined as a front gate into secure networks and eliminates a potential denial-of-service attack surface. Vendors SHOULD clean up tracestate collection on traceparent restart. There are rare cases when the original tracestate entries must be preserved after a restart. This typically happens when the trace-id is reverted back at some point of the trace flow, for instance, when it leaves the secure network. However, it SHOULD be an explicit decision, and not the default behavior.
By default this is probably what Envoy Gateway should configure when proxy tracing is enabled.
I would also go further and suggest that the `traceparent` field propagation is configurable, so that it can conditionally be used depending on the value of the `tracestate` field (or other suitable field).
[optional *Relevant Links*:]
The tracing module for nginx defaults to ignoring the `traceparent` header, and has configuration options for using it in trace propagation: https://nginx.org/en/docs/ngx_otel_module.html#otel_trace_context
I like what Elastic APM does, where it has a configurable trace continuation strategy config option (specifically the `restart_external` option which looks at `tracestate` to determine if the trace came from elastic): https://github.com/elastic/apm/blob/main/specs/agents/trace-continuation.md Something similar could be configurable in Envoy Gateway, to allow you to use the `traceparent` header if a custom `tracestate` flag is set.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.