apache / apache/pinot

Parsing http-headers in RequestContext - broker query event listener

Open
#12,273 10 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

As part of an ongoing discussion in [#12258](https://github.com/apache/pinot/pull/12258#discussion_r1451082198), we would like to gain more inputs from the community on the scope of parsing http-headers in query-request and push it to the event-listener framework.

For additional context, we at Uber want to track extra metadata info related to query-requests such as caller-name/group, tracing-id, queryhash; alongside other query-level metrics like numDocsScanned and E2E latency (already pushed from our event-listener framework). These metadata info might be easy to identify and aggregate certain query patterns.
But we want to avoid processing sensitive information like auth-tokens or other authentication-related fields -- this can be a requirement for other companies as well.

Starting this thread to get consensus from the community on the scope of parsing and how best to implement it.

Few solutions which we have thought of:
- Introduce a broker-level config where we specify a **allowlist of headers** to parse from request and push them to request-context object.
- Introducing a broker-level config to specify a **blocklist of headers** to **exclude** from parsing, pushing all others to the request-context object.

cc @walterddr @ankitsultana

Contributor guide

Open the contributing guide

Research direction

Start with the ongoing discussion in pull request #12258, then compare the proposed allowlist and blocklist approaches for broker request headers. The scope is not decided in this issue; done means reaching community consensus on which headers may be parsed and how they should reach the request-context and event-listener framework.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.