vectordotdev / vectordotdev/vector

NATS source: Allow vector to start with unavailable source

Open
#26,134 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
22.6k
Forks
2.3k
Avg merge
1d 7h
Merged PRs (30d)
146

Description

A note for the community
  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment
Use Cases

Configuring NATS sources, I would still like my vector to run if one of them is unavailable.

Currently, it is able to support a service going down temporary after startup, but it fails to start if one is unavailable.

Attempted Solutions

Searched for any configuration that would allow, but did not find any.

Proposal

I would love to be able to already configure my vector sources, and still have it running and handling logs/metrics/... even if one or more sources are not available at start.

Just retry them at times, until they get available !
(Maybe even include a max delay between retries)

We require it for NATS sources, but the option could probably benefit from something similar for all network sources.

References

No response

Version

0.55

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the NATS source startup path and trace how unavailable sources currently prevent Vector from starting. Compare that behavior with the existing handling for a source that goes down after startup. Done means NATS sources can retry until available while Vector continues processing other configured sources, with any retry-delay behavior covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.