quickwit-oss / quickwit-oss/quickwit

Refactor Source building

Open
#1,064 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement help wanted
Dominant language
Rust
Stars
11.7k
Forks
597
Avg merge
2d 22h
Merged PRs (30d)
37

Description

In #1058, we did one step from (dynamic factory + json) to a static enum world in order to have source type specific source params validation.

We are now half-way between two worlds.

Solution 1

Just go full enum.
The factory is a function of the enum.
Pros: Very simple
Cons: Does not separate the concern super well.

Note: If we go for that route, source_type() should return an enum too.

Solution 2

Use a SourceParams trait that can build a Source type (does the job of the factory) and do dark magic, like what @fmassot did for the IndexConfig deserialization.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading issue #1058 and the IndexConfig deserialization approach mentioned in the discussion. Compare the two proposed Source-building designs and confirm which direction maintainers want; the work is done when one design is selected and the Source factory and source_type behavior are consistently refactored.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.