feat(connectors): Implement QuestDB sink connector over QWP
- Dominant language
- Rust
- Stars
- 4.9k
- Forks
- 432
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 173
Description
### Description
As part of the connector ecosystem roadmap (#2753) I would like to add a native Rust **QuestDB sink** under `core/connectors/sinks/questdb_sink`.
The roadmap lists QuestDB at P3, noting that the InfluxDB connector could cover it via line protocol. That is now out of date: QuestDB 10 introduced **QWP (the QuestDB Wire Protocol)**, a binary columnar protocol over WebSocket which is the native ingestion path for every QuestDB client. ILP remains only for InfluxDB compatibility, so routing through it gives up typed columnar encoding, per-flush acknowledgements, client-side store-and-forward, multi-host failover and durable acknowledgements.
I work on QuestDB, so I can test the Enterprise-only paths directly: TLS, buffering across a failover, and durable ACK.
### Affected area / component
Connectors
### Proposed solution
A `Sink` plugin built on the official [`questdb-rs`](https://crates.io/crates/questdb-rs) crate (Apache-2.0), speaking QWP over WebSocket. One crate serves QuestDB open source and Enterprise, since credentials, TLS, store-and-forward, reconnect and failover are all selected through connect-string keys.
Configuration is a QuestDB connect string plus a small mapping layer (target table, designated-timestamp source, and which payload fields become `SYMBOL` or `UUID` columns), following the `influxdb_sink` / `postgres_sink` conventions.
Delivery is at-least-once, so tables need `DEDUP UPSERT KEYS(...)`. Worth flagging for reviewers: because the runtime commits offsets before `consume()` runs and discards its result (#2928, #2927), a failed batch cannot be replayed from Apache Iggy. QWP's on-disk store-and-forward covers that gap on the write path, which is a concrete reason to prefer it over ILP-over-HTTP here.
Scope is the sink only.
I have a small proof of concept working end to end against QuestDB 10 and QuestDB Enterprise, with unit and `testcontainers` integration coverage. Happy to adjust the design before turning it into a PR.
### Alternatives considered
_No response_
### Contribution
- [x] I'm willing to submit a pull request to implement this feature
### Good first issue
- [ ] I think this could be a good first issue for a new contributor
Contributor guide
Assessment
This issue has not been assessed yet.