AbsaOSS / AbsaOSS/EventGate

Propagate the request correlation id to downstream sinks

Aberta
#212 0 comentários 0 reações 0 responsáveis Ver no GitHub
enhancement
Linguagem predominante
Python
Estrelas
4
Forks
0
Merge médio
1d 5h
PRs com merge (30d)
8

Descrição

### Feature Description

Propagate the request correlation id to the downstream sinks, so consumers of the fan-out targets can join their processing logs to the EventGate request that produced the message:

- **Kafka** — add the id as a record header (e.g. `correlation_id`) in `WriterKafka.write()`. Record headers keep the payload untouched, and consumers can read the header without schema changes.
- **EventBridge** — include the id in the `put_events` entry, either as a top-level field of `Detail` or via `TraceHeader`, in `WriterEventBridge.write()`.
- **Postgres** — optional; would require a schema change (extra column), so likely out of scope for the first iteration.

### Problem / Opportunity

Since #193 / PR #204, every EventGate request resolves a correlation id (`X-Correlation-ID` → `X-Request-ID` → API Gateway request id), logs it on every line and returns it in the `X-Correlation-ID` response header. The id currently stops at EventGate: once a message is fanned out, downstream consumers have no way to correlate their processing with the originating request.

Beneficiaries: teams consuming the Kafka topics and EventBridge events, and anyone debugging a cross-system flow end-to-end — one id would then trace a message from the caller, through EventGate's logs, into every sink's consumer logs.

### Acceptance Criteria

1. A message published to Kafka carries the request's correlation id as a record header; the message payload (value) is byte-for-byte unchanged.
2. An event published to EventBridge carries the request's correlation id in its entry metadata (`TraceHeader` or a documented `Detail` field); the schema-validated payload is unchanged.
3. When no correlation id is resolved (empty id), no header/field is added and writes behave exactly as today.
4. Existing topic schema validation and all existing consumers are unaffected — propagation uses transport metadata only, never the message body.
5. Unit tests cover both writers with and without a bound correlation id; integration tests prove the header/field arrives in the sink.
6. README "Logging & Correlation" documents the downstream propagation contract.

### Proposed Solution

The writers do not currently receive the correlation id; it is bound in `src/utils/observability.py`. Two options:

- a small accessor (e.g. `current_correlation_id()`) in `src/utils/observability.py`, read by the writers, or
- an explicit parameter threaded through `HandlerTopic._write_to_all()` into `Writer.write()`.

The explicit parameter is more testable and keeps writers free of hidden state; the accessor avoids touching the `Writer` interface. Alternative considered and rejected: putting the id into the message body — breaks topic schema validation and changes consumer contracts.

The id is already validated against `^[A-Za-z0-9._:-]{1,128}$` before use, so it is safe to forward verbatim.

### Dependencies / Related

- #193 (structured logging + correlation id)
- PR #204 (implementation; ADR `adr/001-observability/001-observability.md`)

### Additional Context

Follow-up proposed during the second-opinion review of PR #204 (item P7). Postgres propagation can be a separate issue if a schema change is ever justified.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.