FR(tracing): add attribute-based sampling
- Dominant language
- C++
- Stars
- 28.9k
- Forks
- 5.6k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 430
Description
*Title*: Attribute-based trace sampling
*Description*:
In addition to the existing trace sampling configurations (`client`, `random`, and `overall`), it would be useful for Envoy to support some form of sampling based on **request** attributes. This would allow more flexible sampling logic, without the need for custom filters and shared metadata (for example).
This could work similar to the [filters](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/accesslog/v3/accesslog.proto#config-accesslog-v3-accesslogfilter) exposed by access logging (in particular, the [CEL extension filter](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/access_loggers/filters/cel/v3/cel.proto#extensions-access-loggers-filters-cel-v3-expressionfilter)).
On the Istio effort, we've had customers/users ask for functionality that could be addressed by this feature, mainly at gateways and for new requests within a mesh. In particular, these requests have generally been of the following nature:
- sample new requests to certain services at 0% or 100% rate, while all other requests are sampled at the default random rate
- sample incoming requests for certain request paths at 0% or 100%, while all other requests are sampled at the default random rate
A sample proposal for what this _might_ look like in the [HCM.Tracing](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/http_connection_manager/v3/http_connection_manager.proto#extensions-filters-network-http-connection-manager-v3-httpconnectionmanager-tracing) schema:
```protobuf
message Tracing {
message AttributeSampling {
message Sampler {
enum SamplingDecision {
ALWAYS_SAMPLE = 0;
NEVER_SAMPLE = 1;
}
SamplingDecision decision = 1;
repeated string expressions = 2;
}
repeated Sampler samplers = 1;
}
AttributeSampling attribute_sampling = 10;
…
type.v3.Percent client_sampling = 3;
type.v3.Percent random_sampling = 4;
…
}
```
It would operate with a semantic of:
- if a sampling decision has already been made, use that decision.
- if _any_ sampler expression returns `true`, use the configured sampling decision for that sampler.
- if __no__ sampler expression returns `true`, fallback to the `randomSampling` configuration for a sampling decision.
NOTE: this proposal does not support sampling at rates different than 0 or 100 for matches, in an attempt to maintain simplicity. It might be more flexible to replace the `SamplingDecision decision` with a `type.v3.Percent rate` field (at the marginal cost of more complex configuration and implementation).
This might lead to config like:
```yaml
tracing:
attributeSampling:
samplers:
- decision: NEVER_SAMPLE
expressions:
- request.url_path == '/no-trace'
- request.host == 'metadata.internal.svc.cluster.local'
- decision: ALWAYS_SAMPLE
expressions:
- request.url_path == '/login'
- request.host == 'controller.admin.svc.cluster.local'
randomSampling:
value: 1.00
```
If this seemed reasonable and useful to the community, I'd volunteer to handle the implementation.
Contributor guide
Assessment
This issue has not been assessed yet.