vectordotdev / vectordotdev/vector

TCP load balancing & failover

Open
#288 7 comments 11 reactions 1 assignee View on GitHub

@bruceg is already working on this.

Since Aug 12, 2022.

domain: networking domain: reliability have: should meta: feedback needs: requirements sink: socket
Dominant language
Rust
Stars
22.6k
Forks
2.3k
Avg merge
1d 7h
Merged PRs (30d)
146

Description

A common feature across log forwards is the ability to load balance data across a static number of addresses. For example:

[sink.out]
  type = "tcp"
  addresses = ["1.1.1.1:9000", "1.1.1.1:90001"]

Alternatively, DNS could be used for this when support DNS lookup (https://github.com/timberio/vector/issues/252)

Requirements

  • Default load balancing strategy is round robbin.
  • Default to a ttl of 30 seconds, ttl is an option the user can adjust. I think 30 seconds is a good default, especially if TLS is enabled. This will reduce the connection setup overhead. Alternatively, we could leave the connection open and balance over those open connections?
  • As a follow up, I'd like to consider how we'd solve failover. This could be accomplished by setting the ttl to 0? (ex: this image).

Open questions

  1. Does this belong in Vector or is it better solved with DNS or another external system? To me, it seems like a light weight feature that would be useful.
  2. Anything else I'm missing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.