apache / apache/iggy

Add Meilisearch source connector

Open
#3,496 0 comments 0 reactions 0 assignees View on GitHub
connectors enhancement rust
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

## Summary
Add a Meilisearch source connector that polls documents from a Meilisearch index and produces JSON messages into Iggy.

## Motivation
Users should be able to ingest Meilisearch documents into Iggy streams with stable cursor-based polling and persisted connector state.

## Scope
- Add the source connector crate and sample runtime config.
- Document source configuration, retry behavior, and primary-key cursor requirements.
- Add Docker-backed integration coverage for producing indexed documents.

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing source connector crates and their runtime configuration examples, then compare how connector state, polling, retries, and cursors are handled. Add the Meilisearch source connector and sample configuration, document its source settings and primary-key cursor requirements, and run the Docker-backed integration coverage to verify indexed documents become JSON messages in Iggy.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.