feat(request-trace-uploader): add configurable header-based record suppression
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
Why
Some request records must omit request and response payloads before export for a configured set of tenants. WCE needs this for specific NCA IDs. The feature must generalize past NCA IDs, must not remove correlation evidence, and must not leak payloads in later records of the same request.
Scope
- Accept suppression rules as configuration: a header group, a list of match values, and an action. Do not hardcode NCA IDs or a single header name.
- A header group is an ordered set of equivalent header names. Lookup is case-insensitive and resolves to the first alias present on the record.
nvcf-ncaidandnvcf-nca-idare aliases of one group, not two rules. - Support declarable per-group normalization. NCA IDs normalize by stripping the
nca-prefix and-ncasuffix. Match is exact after normalization. - Support scalar and list-valued header forms.
- Apply to record types that carry headers. On Dynamo v1.4.0 that is
request_payloadonly, viapayload.http_request_headers. Make the behavior for record types with no headers explicit: export, drop, or reject at config validation. - Default action
stripPayloadretains only safe correlation metadata, the matched header and value, and an explicit suppression marker. It removes request payload, response payload, and all other headers.dropRecordmust also be selectable. - Carry the decision across later records sharing the request ID when they lack the header.
- Apply suppression only to the upload transport data. Do not claim it removes raw payloads already written to the local spool.
- Emit
records_suppressed_totalby rule and action,records_evaluated_total, andsuppression_header_present_totalby rule, incremented whenever any alias in a rule's header group is present regardless of whether the value matched. Together these must let an operator distinguish a producer not capturing the header from a producer capturing it with no matching values. - Do not halt or drop records when a rule's header is absent. The uploader cannot observe Dynamo's header capture allowlist, and legitimate records will lack the header. Report, do not adjudicate.
- Validate suppression configuration at startup and exit non-zero on unparseable rules, unknown actions, or an empty match value list.
- Never emit header values, request IDs, or payload values as metric attributes, trace attributes, or log fields. Rule names and configured match values are bounded by configuration and may be used as labels.
Tests
- Normalization, duplicate, wrapped ID, empty, and case-sensitivity tests.
- Header alias, header casing, scalar, list, missing, and malformed-header tests.
- Matching request followed by a headerless response test.
- Exact safe-envelope and payload-redaction tests.
- Records with no headers at all under each configured policy.
- Counter behavior proving the three suppression metrics distinguish absent-header from no-match.
- Startup validation rejection cases.
Parent: #1004
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the request-trace-uploader implementation, its startup configuration validation, and the Dynamo v1.4.0 request_payload handling. Trace upload transport, request-ID state across records, and metric emission before working through the listed tests. Done means configurable suppression, safe redaction, explicit no-header behavior, startup rejection, and the three distinguishable counters are covered without claiming to alter the local spool.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, observability, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100