vectordotdev / vectordotdev/vector
TCP load balancing & failover
Open
@bruceg is already working on this.
Since Aug 12, 2022.
domain: networking
domain: reliability
have: should
meta: feedback
needs: requirements
sink: socket
- Dominant language
- Rust
- Stars
- 22.6k
- Forks
- 2.3k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 146
Description
A common feature across log forwards is the ability to load balance data across a static number of addresses. For example:
[sink.out]
type = "tcp"
addresses = ["1.1.1.1:9000", "1.1.1.1:90001"]
Alternatively, DNS could be used for this when support DNS lookup (https://github.com/timberio/vector/issues/252)
Requirements
- Default load balancing strategy is round robbin.
- Default to a
ttlof 30 seconds,ttlis an option the user can adjust. I think 30 seconds is a good default, especially if TLS is enabled. This will reduce the connection setup overhead. Alternatively, we could leave the connection open and balance over those open connections? - As a follow up, I'd like to consider how we'd solve failover. This could be accomplished by setting the
ttlto 0? (ex: this image).
Open questions
- Does this belong in Vector or is it better solved with DNS or another external system? To me, it seems like a light weight feature that would be useful.
- Anything else I'm missing.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.