Bug: Missing juju_model_uuid alert labels and rules

### Bug Description

cos-agent generates alert rules and sends it out over 2 interfaces.
1. downstream-prometheus-scrape: The alerts it sends through this interface which is collected from `monitors` interface (nrpe_application: nrpe) does not have `juju_model_uuid` label in that rule. It does however include that label in alert rules generated locally for itself and collected over other interface like `prometheus-rules`
2. cos-agent: In this interface, cos-proxy does not include all the alert rules that was there in `downstream-prometheus-scrape`. It only sends its own health alert and ignores the rest. (example below)

### To Reproduce

The best way to reproduce is to relate cos-proxy with a charm that can talk to both provider interfaces. I am using opentelemetry-collector.

1. `juju deploy opentelemetry-collector otelco --base ubuntu@22.04 --channel 2/stable`
2. `juju integrate cos-proxy:downstream-prometheus-scrape otelco`
3. `juju integrate cos-proxy:cos-agent otelo`
4. `juju show-unit otelco/14 --format json | jq '.[]."relation-info"[]."application-data"."alert_rules"'`

Use an IDE to prettyfy it

### Environment

Juju: `3.6.8`
Opentelemetry-collecotr: `2/stable`
Base: `ubuntu@22.04`
Cos-proxy: `2/stable`
Cloud: `openstack`

### Relevant log output

```shell
Missing `juju_model_uuid` from NRPE alerts:

"name": "juju_o7k_9f32e6f_ubuntu_0_alert_rules",
"rules": [
{
"alert": "CheckConntrackNrpeAlert",
"expr": "round(avg_over_time(command_status{juju_unit='ubuntu/0',command='check_conntrack'}[15m])) > 1 or ((absent_over_time(command_status{juju_unit='ubuntu/0',command='check_conntrack'}[10m]) == 1))*2 or ((absent_over_time(up{juju_unit='ubuntu/0'}[10m]) == 1))*2",
"for": "0m",
"labels": {
"severity": "{{ if eq $value 0.0 -}} info {{- else if eq $value 1.0 -}} warning {{- else if eq $value 2.0 -}} critical {{- else if eq $value 3.0 -}} error {{- end }}",
"juju_model": "o7k",
"juju_application": "ubuntu",
"juju_unit": "ubuntu/0",
"nrpe_application": "nrpe",
"nrpe_unit": "nrpe/0"
},
"annotations": {
"summary": "Unit {{ $labels.juju_unit }}: {{ $labels.command }} {{ $labels.severity }}.",
"description": "Check provided by nrpe_exporter in model {{ $labels.juju_model }} is failing.\nFailing check = {{ $labels.command }}\nUnit = {{ $labels.juju_unit }}\nValue = {{ $value }}\nLegend:\n - StatusOK = 0\n - StatusWarning = 1\n - StatusCritical = 2\n - StatusUnknown = 3"
}
},
{
"alert": "CheckSystemdScopesNrpeAlert",
"expr": "round(avg_over_time(command_status{juju_unit='ubuntu/0',command='check_systemd_scopes'}[15m])) > 1 or ((absent_over_time(command_status{juju_unit='ubuntu/0',command='check_systemd_scopes'}[10m]) == 1))*2 or ((absent_over_time(up{juju_unit='ubuntu/0'}[10m]) == 1))*2",
"for": "0m",
"labels": {
"severity": "{{ if eq $value 0.0 -}} info {{- else if eq $value 1.0 -}} warning {{- else if eq $value 2.0 -}} critical {{- else if eq $value 3.0 -}} error {{- end }}",
"juju_model": "o7k",
"juju_application": "ubuntu",
"juju_unit": "ubuntu/0",
"nrpe_application": "nrpe",
"nrpe_unit": "nrpe/0"
},
"annotations": {
"summary": "Unit {{ $labels.juju_unit }}: {{ $labels.command }} {{ $labels.severity }}.",
"description": "Check provided by nrpe_exporter in model {{ $labels.juju_model }} is failing.\nFailing check = {{ $labels.command }}\nUnit = {{ $labels.juju_unit }}\nValue = {{ $value }}\nLegend:\n - StatusOK = 0\n - StatusWarning = 1\n - StatusCritical = 2\n - StatusUnknown = 3"
}
},
{
"alert": "CheckLoadNrpeAlert",
"expr": "round(avg_over_time(command_status{juju_unit='ubuntu/0',command='check_load'}[15m])) > 1 or ((absent_over_time(command_status{juju_unit='ubuntu/0',command='check_load'}[10m]) == 1))*2 or ((absent_over_time(up{juju_unit='ubuntu/0'}[10m]) == 1))*2",
"for": "0m",
"labels": {
"severity": "{{ if eq $value 0.0 -}} info {{- else if eq $value 1.0 -}} warning {{- else if eq $value 2.0 -}} critical {{- else if eq $value 3.0 -}} error {{- end }}",
"juju_model": "o7k",
"juju_application": "ubuntu",
"juju_unit": "ubuntu/0",
"nrpe_application": "nrpe",
"nrpe_unit": "nrpe/0"
},
"annotations": {
"summary": "Unit {{ $labels.juju_unit }}: {{ $labels.command }} {{ $labels.severity }}.",
"description": "Check provided by nrpe_exporter in model {{ $labels.juju_model }} is failing.\nFailing check = {{ $labels.command }}\nUnit = {{ $labels.juju_unit }}\nValue = {{ $value }}\nLegend:\n - StatusOK = 0\n - StatusWarning = 1\n - StatusCritical = 2\n - StatusUnknown = 3"
}
},
{
"alert": "CheckSwapActivityNrpeAlert",
"expr": "round(avg_over_time(command_status{juju_unit='ubuntu/0',command='check_swap_activity'}[15m])) > 1 or ((absent_over_time(command_status{juju_unit='ubuntu/0',command='check_swap_activity'}[10m]) == 1))*2 or ((absent_over_time(up{juju_unit='ubuntu/0'}[10m]) == 1))*2",
"for": "0m",
"labels": {
"severity": "{{ if eq $value 0.0 -}} info {{- else if eq $value 1.0 -}} warning {{- else if eq $value 2.0 -}} critical {{- else if eq $value 3.0 -}} error {{- end }}",
"juju_model": "o7k",
"juju_application": "ubuntu",
"juju_unit": "ubuntu/0",
"nrpe_application": "nrpe",
"nrpe_unit": "nrpe/0"
},
"annotations": {
"summary": "Unit {{ $labels.juju_unit }}: {{ $labels.command }} {{ $labels.severity }}.",
"description": "Check provided by nrpe_exporter in model {{ $labels.juju_model }} is failing.\nFailing check = {{ $labels.command }}\nUnit = {{ $labels.juju_unit }}\nValue = {{ $value }}\nLegend:\n - StatusOK = 0\n - StatusWarning = 1\n - StatusCritical = 2\n - StatusUnknown = 3"
}
},
{
"alert": "CheckMemNrpeAlert",
"expr": "round(avg_over_time(command_status{juju_unit='ubuntu/0',command='check_mem'}[15m])) > 1 or ((absent_over_time(command_status{juju_unit='ubuntu/0',command='check_mem'}[10m]) == 1))*2 or ((absent_over_time(up{juju_unit='ubuntu/0'}[10m]) == 1))*2",
"for": "0m",
"labels": {
"severity": "{{ if eq $value 0.0 -}} info {{- else if eq $value 1.0 -}} warning {{- else if eq $value 2.0 -}} critical {{- else if eq $value 3.0 -}} error {{- end }}",
"juju_model": "o7k",
"juju_application": "ubuntu",
"juju_unit": "ubuntu/0",
"nrpe_application": "nrpe",
"nrpe_unit": "nrpe/0"
},
"annotations": {
"summary": "Unit {{ $labels.juju_unit }}: {{ $labels.command }} {{ $labels.severity }}.",
"description": "Check provided by nrpe_exporter in model {{ $labels.juju_model }} is failing.\nFailing check = {{ $labels.command }}\nUnit = {{ $labels.juju_unit }}\nValue = {{ $value }}\nLegend:\n - StatusOK = 0\n - StatusWarning = 1\n - StatusCritical = 2\n - StatusUnknown = 3"
}
},
{
"alert": "CheckArpCacheNrpeAlert",
"expr": "round(avg_over_time(command_status{juju_unit='ubuntu/0',command='check_arp_cache'}[15m])) > 1 or ((absent_over_time(command_status{juju_unit='ubuntu/0',command='check_arp_cache'}[10m]) == 1))*2 or ((absent_over_time(up{juju_unit='ubuntu/0'}[10m]) == 1))*2",
"for": "0m",
"labels": {
"severity": "{{ if eq $value 0.0 -}} info {{- else if eq $value 1.0 -}} warning {{- else if eq $value 2.0 -}} critical {{- else if eq $value 3.0 -}} error {{- end }}",
"juju_model": "o7k",
"juju_application": "ubuntu",
"juju_unit": "ubuntu/0",
"nrpe_application": "nrpe",
"nrpe_unit": "nrpe/0"
},
"annotations": {
"summary": "Unit {{ $labels.juju_unit }}: {{ $labels.command }} {{ $labels.severity }}.",
"description": "Check provided by nrpe_exporter in model {{ $labels.juju_model }} is failing.\nFailing check = {{ $labels.command }}\nUnit = {{ $labels.juju_unit }}\nValue = {{ $value }}\nLegend:\n - StatusOK = 0\n - StatusWarning = 1\n - StatusCritical = 2\n - StatusUnknown = 3"
}
},
{
"alert": "CheckRoFilesystemNrpeAlert",
"expr": "round(avg_over_time(command_status{juju_unit='ubuntu/0',command='check_ro_filesystem'}[15m])) > 1 or ((absent_over_time(command_status{juju_unit='ubuntu/0',command='check_ro_filesystem'}[10m]) == 1))*2 or ((absent_over_time(up{juju_unit='ubuntu/0'}[10m]) == 1))*2",
"for": "0m",
"labels": {
"severity": "{{ if eq $value 0.0 -}} info {{- else if eq $value 1.0 -}} warning {{- else if eq $value 2.0 -}} critical {{- else if eq $value 3.0 -}} error {{- end }}",
"juju_model": "o7k",
"juju_application": "ubuntu",
"juju_unit": "ubuntu/0",
"nrpe_application": "nrpe",
"nrpe_unit": "nrpe/0"
},
"annotations": {
"summary": "Unit {{ $labels.juju_unit }}: {{ $labels.command }} {{ $labels.severity }}.",
"description": "Check provided by nrpe_exporter in model {{ $labels.juju_model }} is failing.\nFailing check = {{ $labels.command }}\nUnit = {{ $labels.juju_unit }}\nValue = {{ $value }}\nLegend:\n - StatusOK = 0\n - StatusWarning = 1\n - StatusCritical = 2\n - StatusUnknown = 3"
}
},
{
"alert": "CheckSpaceRootNrpeAlert",
"expr": "round(avg_over_time(command_status{juju_unit='ubuntu/0',command='check_space_root'}[15m])) > 1 or ((absent_over_time(command_status{juju_unit='ubuntu/0',command='check_space_root'}[10m]) == 1))*2 or ((absent_over_time(up{juju_unit='ubuntu/0'}[10m]) == 1))*2",
"for": "0m",
"labels": {
"severity": "{{ if eq $value 0.0 -}} info {{- else if eq $value 1.0 -}} warning {{- else if eq $value 2.0 -}} critical {{- else if eq $value 3.0 -}} error {{- end }}",
"juju_model": "o7k",
"juju_application": "ubuntu",
"juju_unit": "ubuntu/0",
"nrpe_application": "nrpe",
"nrpe_unit": "nrpe/0"
},
"annotations": {
"summary": "Unit {{ $labels.juju_unit }}: {{ $labels.command }} {{ $labels.severity }}.",
"description": "Check provided by nrpe_exporter in model {{ $labels.juju_model }} is failing.\nFailing check = {{ $labels.command }}\nUnit = {{ $labels.juju_unit }}\nValue = {{ $value }}\nLegend:\n - StatusOK = 0\n - StatusWarning = 1\n - StatusCritical = 2\n - StatusUnknown = 3"
}
},
{
"alert": "CheckRebootNrpeAlert",
"expr": "round(avg_over_time(command_status{juju_unit='ubuntu/0',command='check_reboot'}[15m])) > 1 or ((absent_over_time(command_status{juju_unit='ubuntu/0',command='check_reboot'}[10m]) == 1))*2 or ((absent_over_time(up{juju_unit='ubuntu/0'}[10m]) == 1))*2",
"for": "0m",
"labels": {
"severity": "{{ if eq $value 0.0 -}} info {{- else if eq $value 1.0 -}} warning {{- else if eq $value 2.0 -}} critical {{- else if eq $value 3.0 -}} error {{- end }}",
"juju_model": "o7k",
"juju_application": "ubuntu",
"juju_unit": "ubuntu/0",
"nrpe_application": "nrpe",
"nrpe_unit": "nrpe/0"
},
"annotations": {
"summary": "Unit {{ $labels.juju_unit }}: {{ $labels.command }} {{ $labels.severity }}.",
"description": "Check provided by nrpe_exporter in model {{ $labels.juju_model }} is failing.\nFailing check = {{ $labels.command }}\nUnit = {{ $labels.juju_unit }}\nValue = {{ $value }}\nLegend:\n - StatusOK = 0\n - StatusWarning = 1\n - StatusCritical = 2\n - StatusUnknown = 3"
}
}

------------------------------------------------------------------------------------------

Alerts sent out using cos-agent:

{
"name": "o7k_9f32e6f8_cos_proxy_HostHealth_alerts",
"rules": [
{
"alert": "HostDown",
"expr": "up < 1",
"for": "5m",
"labels": {
"severity": "critical",
"juju_model": "o7k",
"juju_model_uuid": "9f32e6f8-ac3d-436b-87bb-2cc4c2678cd2",
"juju_application": "cos-proxy",
"juju_charm": "cos-proxy"
},
"annotations": {
"summary": "Host '{{ $labels.instance }}' is down.",
"description": "Host '{{ $labels.instance }}' is down, failed to scrape.\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
}
},
{
"alert": "HostMetricsMissing",
"expr": "absent(up)",
"for": "5m",
"labels": {
"severity": "critical",
"juju_model": "o7k",
"juju_model_uuid": "9f32e6f8-ac3d-436b-87bb-2cc4c2678cd2",
"juju_application": "cos-proxy",
"juju_charm": "cos-proxy"
},
"annotations": {
"summary": "Metrics not received from host '{{ $labels.instance }}', failed to remote write.",
"description": "Metrics not received from host '{{ $labels.instance }}', failed to remote write.\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
}
}
]
},
```

### Additional context

Let me know if you need more details.

Start by reproducing the issue with the listed Juju deploy/integrate commands and inspect the otelco application-data alert_rules using juju show-unit. Compare rules from downstream-prometheus-scrape with those sent through cos-agent, focusing on juju_model_uuid and omitted rules. Done means downstream alerts include juju_model_uuid and cos-agent forwards the complete alert-rule set.

Written by the indexing model from the issue text.