Grafana-agent down in dashboard status

#### Description
Grafana Agent shows down in dashboard status - newly deployed.

#### Screenshots


#### Log Messages
Grafana-agent logs (all periodically with similar error):
```
ts=2024-09-27T02:09:44.980753674Z level=error msg="non-recoverable error" component=prometheus.remote_write.default subcomponent=rw remote_name=bd527e url=http://prometheus.coder-observability.svc/api/v1/write count=2000 exemplarCount=0 err="server returned HTTP status 400 Bad Request: out of order sample"
ts=2024-09-27T02:09:50.086696001Z level=error msg="non-recoverable error" component=prometheus.remote_write.default subcomponent=rw remote_name=bd527e url=http://prometheus.coder-observability.svc/api/v1/write count=1558 exemplarCount=0 err="server returned HTTP status 400 Bad Request: out of order sample"
ts=2024-09-27T02:10:05.040293177Z level=error msg="non-recoverable error" component=prometheus.remote_write.default subcomponent=rw remote_name=bd527e url=http://prometheus.coder-observability.svc/api/v1/write count=626 exemplarCount=0 err="server returned HTTP status 400 Bad Request: out of order sample"
ts=2024-09-27T02:10:44.998642978Z level=error msg="non-recoverable error" component=prometheus.remote_write.default subcomponent=rw remote_name=bd527e url=http://prometheus.coder-observability.svc/api/v1/write count=2000 exemplarCount=0 err="server returned HTTP status 400 Bad Request: out of order sample"
ts=2024-09-27T02:11:05.079034516Z level=error msg="non-recoverable error" component=prometheus.remote_write.default subcomponent=rw remote_name=bd527e url=http://prometheus.coder-observability.svc/api/v1/write count=626 exemplarCount=0 err="server returned HTTP status 400 Bad Request: out of order sample"
```

Grafana pod logs:
```
logger=context userId=0 orgId=0 uname= t=2024-09-27T02:41:46.069582049Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=10.0.22.6 time_ms=0 duration=888.896µs size=105 referer= handler=/api/live/ws status_source=server
logger=context userId=0 orgId=0 uname= t=2024-09-27T02:42:07.000216216Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=10.0.22.6 time_ms=0 duration=759.171µs size=105 referer= handler=/api/live/ws status_source=server
logger=provisioning.dashboard type=file name=sidecar t=2024-09-27T02:42:13.869036352Z level=error msg="Cannot read directory" error="stat /tmp/dashboards: no such file or directory"
logger=provisioning.dashboard type=file name=sidecar t=2024-09-27T02:42:13.869120292Z level=error msg="Failed to read content of symlinked path" path=/tmp/dashboards error="lstat /tmp/dashboards: no such file or directory"
logger=provisioning.dashboard type=file name=sidecar t=2024-09-27T02:42:13.869133508Z level=info msg="falling back to original path due to EvalSymlink/Abs failure"
logger=provisioning.dashboard type=file name=sidecar t=2024-09-27T02:42:13.869144257Z level=error msg="failed to search for dashboards" error="stat /tmp/dashboards: no such file or directory"
logger=context userId=0 orgId=0 uname= t=2024-09-27T02:42:27.003993831Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=10.0.22.6 time_ms=0 duration=667.764µs size=105 referer= handler=/api/live/ws status_source=server
logger=provisioning.dashboard type=file name=sidecar t=2024-09-27T02:42:43.868560334Z level=error msg="Cannot read directory" error="stat /tmp/dashboards: no such file or directory"
logger=provisioning.dashboard type=file name=sidecar t=2024-09-27T02:42:43.868611097Z level=error msg="Failed to read content of symlinked path" path=/tmp/dashboards error="lstat /tmp/dashboards: no such file or directory"
logger=provisioning.dashboard type=file name=sidecar t=2024-09-27T02:42:43.868620434Z level=info msg="falling back to original path due to EvalSymlink/Abs failure"
logger=provisioning.dashboard type=file name=sidecar t=2024-09-27T02:42:43.86862969Z level=error msg="failed to search for dashboards" error="stat /tmp/dashboards: no such file or directory"
```

#### To Reproduce
```
helm install coder-observability coder-observability/coder-observability \
--version 0.2.1 \
--namespace coder-observability \
--values values.yaml
```
* Note: Can share the values.yaml - latest and with updates only to postgres.