block / block/proto-fleet

[Notifications] Monitoring sidecar in the docker-compose stack

Open
#174 0 comments 0 reactions 1 assignee Claimed by @illegalprime View on GitHub
Dominant language
Go
Stars
55
Forks
16
Avg merge
1d 17h
Merged PRs (30d)
84

Description

Stand up the off-the-shelf alerting pipeline as internal sidecars next to `fleet-api`. The stack is invisible to the host; ProtoFleet is the only client. Turning the stack off must leave the rest of the fleet working.

Checklist:

- [ ] Add `otel-collector`, `victoria-metrics` (with vmalert), and `alertmanager` services to `deployment-files/docker-compose.yaml` and `server/docker-compose.base.yaml`. Attach them and `fleet-api` to a new `monitoring` docker network. No host-bound `ports:` — `expose:` only.
- [ ] `FLEET_NOTIFICATIONS_ENABLED` env (default `true`) gates sidecar startup and the notifications surface in the API.
- [ ] Provision starter configs at `deployment-files/server/monitoring/{otel-collector.yaml,vmalert/rules.yml,alertmanager/alertmanager.yml}`. Mount them read-write so the reload pipeline can swap them.
- [ ] Wire vmalert to alertmanager and victoria-metrics; expose Alertmanager `/-/reload` and vmalert `/-/reload` reachable only from `fleet-api`.
- [ ] Built-in `protofleet-self` rule group (always loaded, undeletable from the API): `OTelCollectorDown`, `VMAlertEvaluationStalled`, `AlertmanagerUnreachable`, `MonitoringStackDegraded` with a 5-minute `for:`. Self alerts route to a built-in `__protofleet_internal` receiver that always logs.
- [ ] Update `deployment-files/README.md` with the new sidecars.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.