quickwit-oss / quickwit-oss/quickwit
Create utility scripts for setting up and loading common ingestion sources for testing.
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Currently, It's very simple to test Quickwit locally using a basic source file and a file input. But creating and testing sources for things like Kafka and Kinesis take more time and involve a lot of repeated steps which can be simplified into a common set of scripts.
Hopefully, this should save people from repeating the same set of steps over and over again and also save the extra leg work required by each person trying to test locally for each source.
For example, we could have a script that setup Kafka or Kinesis and pre-loads some data on them to be indexed by Quickwit.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing local testing workflow for a basic source file and file input, then identify the repeated setup and data-loading steps for Kafka and Kinesis. Done means common utility scripts can set up those sources and preload data for indexing by Quickwit during local tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka
- Domain
- data-engineering, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100