datafusion-contrib / datafusion-contrib/datafusion-distributed
Decouple WorkerChannel from http headers
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 139
- Forks
- 67
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 35
Description
Follow up on:
The PR introduced the WorkerChannel abstraction for decoupling gRPC from the project's core logic. This way people can bring their own transport protocols.
The WorkerChannel is still coupled to http::HeadersThough mainly for two reasons:
- Transporting a serialized version of the
ConfigOptions injected by the user - Passing through headers across worker jumps
This probably should be factored out of the core logic of this project into the gRPC specific implementation, or maybe some other module that more implementations can use, not just gRPC.
Before anyone taking this issue, please feel free to post the approach here, as I'd expect this to need some discussion.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the WorkerChannel abstraction introduced in PR 512 and trace how ConfigOptions and headers move through the core logic and gRPC implementation. Propose an approach in this issue before coding; done means the core WorkerChannel no longer depends on HTTP headers while the existing gRPC behavior remains supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grpc, rust
- Domain
- distributed-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100