vectordotdev / vectordotdev/vector

Building topology from scripts

Open
#1,328 3 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

domain: config domain: setup domain: topology meta: idea needs: approval needs: requirements
Dominant language
Rust
Stars
22.6k
Forks
2.3k
Avg merge
1d 7h
Merged PRs (30d)
146

Description

It is interesting to explore ways to build topology from a scripting language, such as Lua or JavaScript.

Initially we can experiment with it without modifying Vector's code by implementing APIs for one of scripting languages which would allow to generate TOML configs and then send SIGHUP signal to Vector to load it (or just restart it). The overall idea of implementing it would be similar to the implementation of Vector kernel for Jupyter (https://github.com/timberio/vector/issues/1318).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the linked Jupyter kernel issue (#1318) and compare its approach with the proposed scripting APIs. Explore how a Lua or JavaScript API could generate TOML configurations and trigger Vector reloads with SIGHUP or a restart. Done means a concrete, working approach for building topology from scripts.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, lua
Domain
stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.