self-deployed rule-evaluator "One or more TimeSeries could not be written"

Hi, I currently have deployed the self-managed rule-evaluator. We have seen the error shown below for all of our recording rules since our migration to GMP.
```json
{
"jsonPayload": {
"caller": "export.go:946",
"size": 5,
"msg": "send batch",
"level": "error",
"ts": "2024-07-18T13:56:45.907003101Z",
"err": "rpc error: code = InvalidArgument desc = One or more TimeSeries could not be written: One or more points were written more frequently than the maximum sampling period configured for the metric.: prometheus_target{location:xxx,cluster:xxx,instance:,job:,namespace:xxx} timeSeries[0-4]: prometheus.googleapis.com/slo:current_burn_rate:ratio/gauge{owner:xxx,tenant_id:xxx,sloth_window:5m,sloth_service:rest,sloth_id:rest-best-effort-availability,sloth_slo:best-effort-availability}\nerror details: name = Unknown desc = total_point_count:5 success_point_count:2 errors:{status:{code:9} point_count:3}"
},
"timestamp": "2024-07-18T13:56:45.907648378Z",
"severity": "ERROR",
"labels": {
"k8s-pod/gmp/scrape": "1m",
},
}
```
This error, multiplied by our number of recording rules, produces several thousand of these errors per hour.

Here is a sample recording rule:
```yaml
groups:
- name: app_overview
rules:
- record: route_rate:app_response_count:sum5m
expr: sum(rate(app_response_count[5m])) by (route)
```
Modifying how we aggregate the metric, to include project_id, location and namespace will make the error go away. However, this means every recording rule we have/create going forward will need to be mindful and include this.

Our rule-evaluator config is relatively simple:
```yaml
global:
external_labels: {} # we have a few labels here
evaluation_interval: 60s
rule_files: []
```
As well as the args we pass at runtime
```yaml
args:
- "--config.file=/prometheus/config_out/config.yaml"
- "--web.listen-address=:9092"
```

I have tried leveraging the `external_labels` property here to ensure project_id, location, and namespace are present however that made the error more frequent.

Additional context:

- we deploy a single rule evaluator per project, we have multiple projects, the error occurs in each
- these metrics all originate from a single namespace, rule-evaluator is deployed to a different namespace
- we are using the managed collector, the error appears to only come from recording rules.

If there any additional information that will be helpful, please let me know.