ADORSYS-GIS / ADORSYS-GIS/lightbridge-authz

[Story]: Export lightbridge-authz telemetry to the existing Alloy OTLP receiver

Ouverte
#639 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
user-story
Langage dominant
Rust
Étoiles
0
Forks
1
Merge moyen
7 h 7 min
PR mergées (30 j)
237

Description

## User Story

**As** whoever is on call for `auth.ai.camer.digital`,
**I want** `lightbridge-authz`'s traces and metrics to reach the observability stack,
**so that** auth failures can be seen in Grafana rather than reconstructed from `kubectl logs`.

## Source of truth (links)

Parent epic #638. Motivating incident: #623, a 3h32m fresh-login outage nothing detected.

## Context

`lightbridge-authz` is instrumented with OpenTelemetry already (`lightbridge_authz_core::tracing`, OTLP → Jaeger in local compose). In prod it exports nowhere: there is no collector for it, and it is the only major service without one.

**Use the existing Alloy DaemonSet — not a dedicated OTel collector.** Decision by the repo owner. `governance-ai-cli`, `copilot` and `keycloak` each run their own collector; that pattern is explicitly *not* being copied.

Alloy already does the work. From the live `observability/alloy` ConfigMap:

```alloy
otelcol.receiver.otlp "default" {
grpc { endpoint = "0.0.0.0:4317" }
http { endpoint = "0.0.0.0:4318" }
output {
traces = [otelcol.exporter.otlp.tempo.input]
logs = [otelcol.exporter.loki.default.input]
metrics = [otelcol.exporter.prometheus.default.input]
}
}
```

So this is a **configuration change, not a deployment**: point the exporter at Alloy and the fan-out to Tempo/Loki/Mimir already exists.

## Acceptance Criteria

- [ ] `api`, `opa`, `idp` and `budget` export OTLP to the Alloy DaemonSet endpoint, configured in `ai-helm-values` (not hardcoded).
- [ ] Traces for a login appear in **Tempo**, queryable by trace ID.
- [ ] Metrics appear in **Mimir**, labelled well enough to distinguish the four components.
- [ ] Reaching Alloy from the `converse` namespace is **verified**, not assumed — the epic flags this as an unconfirmed assumption. If a NetworkPolicy or the DaemonSet's node-local addressing blocks it, that finding is the story's output and the approach gets revisited rather than forced.
- [ ] `lightbridge-authz` logs are confirmed present in Loki with the labels a dashboard can filter on (`namespace`, `pod`, `container`), by running a real LogQL query and pasting the result.
- [ ] No new collector Deployment/DaemonSet is added.

## Out of Scope

Dashboards (story 2) and alerts (story 3). New instrumentation in Rust — this story ships what is already emitted.

## Test Plan / Verification evidence

Perform a real login against staging or prod, then produce:
1. its trace in Tempo,
2. a LogQL query returning that request's log lines,
3. `kubectl` output showing no new collector workload.

## AI Usage Declaration

AI (Claude Opus 5, Claude Code) surveyed the deployed stack and drafted this story.

- [x] Alloy's OTLP receiver was read from the live ConfigMap, not assumed from documentation — which is what reduces this story from "deploy a collector" to "point an exporter".
- [x] The reachability assumption is stated as unverified rather than presented as fact.

## Human verification completed

- [ ] Owner confirms Alloy-only and the config-not-code placement.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Start with the live observability/alloy ConfigMap and the ai-helm-values configuration for api, opa, idp, and budget; confirm the existing Alloy OTLP receiver endpoints before changing values. Verify connectivity from the converse namespace rather than assuming it, then perform a real staging or production login and check Tempo, Mimir, and Loki with the requested trace, LogQL result, and kubectl output showing no new collector workload. Done means all four components export through Alloy and the reachability result is recorded.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
grafana, helm, kubernetes, rust
Domaine
devops, infrastructure, observability-sre
Type d'issue
Fonctionnalité
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Active
Clarté
Clairement spécifiée
Accessibilité débutants
45/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.