AbsaOSS / AbsaOSS/EventGate

Propagate the request correlation id to downstream sinks

未关闭
#212 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
enhancement
主要语言
Python
星标
4
派生
0
平均合并
1 天 5 小时
30 天内合并 PR
8

描述

### Feature Description

Propagate the request correlation id to the downstream sinks, so consumers of the fan-out targets can join their processing logs to the EventGate request that produced the message:

- **Kafka** — add the id as a record header (e.g. `correlation_id`) in `WriterKafka.write()`. Record headers keep the payload untouched, and consumers can read the header without schema changes.
- **EventBridge** — include the id in the `put_events` entry, either as a top-level field of `Detail` or via `TraceHeader`, in `WriterEventBridge.write()`.
- **Postgres** — optional; would require a schema change (extra column), so likely out of scope for the first iteration.

### Problem / Opportunity

Since #193 / PR #204, every EventGate request resolves a correlation id (`X-Correlation-ID` → `X-Request-ID` → API Gateway request id), logs it on every line and returns it in the `X-Correlation-ID` response header. The id currently stops at EventGate: once a message is fanned out, downstream consumers have no way to correlate their processing with the originating request.

Beneficiaries: teams consuming the Kafka topics and EventBridge events, and anyone debugging a cross-system flow end-to-end — one id would then trace a message from the caller, through EventGate's logs, into every sink's consumer logs.

### Acceptance Criteria

1. A message published to Kafka carries the request's correlation id as a record header; the message payload (value) is byte-for-byte unchanged.
2. An event published to EventBridge carries the request's correlation id in its entry metadata (`TraceHeader` or a documented `Detail` field); the schema-validated payload is unchanged.
3. When no correlation id is resolved (empty id), no header/field is added and writes behave exactly as today.
4. Existing topic schema validation and all existing consumers are unaffected — propagation uses transport metadata only, never the message body.
5. Unit tests cover both writers with and without a bound correlation id; integration tests prove the header/field arrives in the sink.
6. README "Logging & Correlation" documents the downstream propagation contract.

### Proposed Solution

The writers do not currently receive the correlation id; it is bound in `src/utils/observability.py`. Two options:

- a small accessor (e.g. `current_correlation_id()`) in `src/utils/observability.py`, read by the writers, or
- an explicit parameter threaded through `HandlerTopic._write_to_all()` into `Writer.write()`.

The explicit parameter is more testable and keeps writers free of hidden state; the accessor avoids touching the `Writer` interface. Alternative considered and rejected: putting the id into the message body — breaks topic schema validation and changes consumer contracts.

The id is already validated against `^[A-Za-z0-9._:-]{1,128}$` before use, so it is safe to forward verbatim.

### Dependencies / Related

- #193 (structured logging + correlation id)
- PR #204 (implementation; ADR `adr/001-observability/001-observability.md`)

### Additional Context

Follow-up proposed during the second-opinion review of PR #204 (item P7). Postgres propagation can be a separate issue if a schema change is ever justified.

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。