Create config setting to customize an environment for querying
- Dominant language
- Go
- Stars
- 11.7k
- Forks
- 802
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 80
Description
## The problem
When we query profiles for applications that are very large, it's pertinent to add additional context to the query to scope it down to query a more realistic result set. There's a couple different strategies that can achieve this end:
1. Use relabeling to synthesize this label `service_name=/`. The `namespace` prefix ensures we restrict the query to a reasonable subset of profiles (this is what we do today).
2. Manually add another label to the query to ensure the result set is manageable. E.g.
```
{namespace="", service_name=""}
```
Both of these options work, but have some short-comings. In option 1, we are forcing users to use relabel rules to make Pyroscope usable at scale. In option 2, we are requiring that users have an understanding that they need specific labels in order to create a query that is realistic to execute.
The long and short is, both options reduce Pyroscope's out-of-the-box usability. There's additional know-how and semi-arcane knowledge that is required before Pyroscope can be used at scale.
## A solution
To help bridge this usability gap, we can create a mandatory label that is used to qualify all queries. This label can be configurable through our configuration file and we can provide a reasonable default value to allow users who aren't dealing with large scale applications to avoid having to worry about such details.
To better illustrate this point, we could have two additional configuration values:
- `querier.environment_label.name` - Sets the mandatory label name for every query (example: `"namespace"`)
- `querier.environment_label.default_value` - Sets the default label value if the mandatory label is not provided (example: `"profiles-ops-001"` or `""`)
This would require all queries to either:
- Provide a `namespace=` value with every query
- Use `namespace=` if no `namespace` label was provided in the query
### Alternative
As Cyril [pointed out](https://github.com/grafana/pyroscope/issues/2796#issuecomment-1838084350), this could be managed entirely within the UI. This should help reduce complexity in the Pyroscope server configuration.
## Considerations
This should help improve a usability concern of Pyroscope and allow every users to customize Pyroscope queries to respect their environments, without having to go through extra hoops of relabeling or knowing specific labels that always need to be added (but aren't required) to reduce the result set to a realistic size.
That said, there are concerns about whether this approach is flexible enough. Certainly there may be some customers who have more complicated use cases which this will not full cover. Additionally, we need to consider the UX when designing queries. We need to make it obvious that a specific label must be specified or a default value will be used. We also should accommodate users wanting to opt out of this feature entirely.
```[tasklist]
### Tasks
- [ ] https://github.com/grafana/pyroscope/pull/2711
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.