vectordotdev / vectordotdev/vector

Support Sonic integration in addition to Elastic

Open
#988 5 comments 9 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

domain: logs domain: sinks needs: approval needs: requirements sink: new
Dominant language
Rust
Stars
22.6k
Forks
2.3k
Avg merge
1d 7h
Merged PRs (30d)
146

Description

Sonic is a fast, lightweight and schema-less search backend. It ingests search texts and identifier tuples that can then be queried against in a microsecond's time. It implemented in Rust too, which makes it a perfect complement to Vector.

Sonic can be used as a simple alternative to super-heavy and full-featured search backends such as Elasticsearch in some use-cases. It is capable of normalizing natural language search queries, auto-completing a search query and providing the most relevant results for a query. Sonic is an identifier index, rather than a document index; when queried, it returns IDs that can then be used to refer to the matched documents in an external database.

A strong attention to performance and code cleanliness has been given when designing Sonic. It aims at being crash-free, super-fast and puts minimum strain on server resources (our measurements have shown that Sonic - when under load - responds to search queries in the μs range, eats ~30MB RAM and has a low CPU footprint; see our benchmarks).

image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by reviewing Vector’s existing Elasticsearch integration and the Sonic API documentation, then determine the required configuration, ingestion, and query behavior; done means Sonic can be used as an alternative search backend with coverage for the supported behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, rust
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.