Lack of documentation about cluster networking
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
While setting up a Pinot cluster in an environment with strict egress/ingress rules it has been a challenge to bump into various cluster interop that wasn't documented.
Based on the docs I recall, the only concise indication of the connectivity is this diagram: https://docs.pinot.apache.org/basics/architecture#core-components
This came with a couple surprises so far:
* Servers connect to the Controller's API for pulling locally stored segments
* Controllers proxy queries from the query console to the Broker's API
There's other areas that I'm lightly aware of but have not yet dealt with, such as batch ingestion jobs connecting to Controllers for notifying of new segments.
Contributor guide
Research direction
Start with the cluster architecture page at docs.pinot.apache.org/basics/architecture#core-components and review the connectivity examples in this issue. Document the relevant cluster network paths, including server-to-Controller segment access, Controller-to-Broker query proxying, and batch-ingestion connections to Controllers.
Written by the indexing model from the issue text.
Assessment
- Domain
- distributed-systems, documentation, networking
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100