connectors: add a sink recovery conformance suite
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- rust
- Domain
- distributed-systems, testing
Research direction
Begin fixture development with #4233, then read the related implementations in #4234, #4235, and #4236 and inspect the actual HTTP sink and its acknowledgment boundaries. Done means the reusable suite covers the listed recovery scenarios, runs in CI, and supported plugins have coverage without missing destination records.
Written by the indexing model from the issue text.
Description
Parent tracker: #4232.
Build reusable failure fixtures and assertions for the agreed delivery contract. Verify destination record identities and saved recovery positions independently of the runtime's processing counters.
Required scenarios:
- Rejected delivery followed by restart and successful replay through the actual HTTP sink.
- Destination success followed by abrupt termination before checkpoint completion. Duplicates are permitted; missing records are not.
- Buffered acceptance, partial delivery, and failed flush followed by recovery.
- A failed record followed by later completion, including multiple partitions.
- Checkpoint timeouts where the server did and did not persist the update.
- Restart with delayed completions from the previous instance, and ownership changes where supported by the agreed contract.
- Decode and transform errors, intentional filtering, and any supported dead-letter policy.
- Incompatible plugins and unavailable source records during recovery.
Use controlled synchronization and fault injection to target the relevant boundaries. Include abrupt process termination so graceful shutdown cannot hide missing delivery barriers.
The common suite must run in CI, and supported plugins must have coverage for their actual acknowledgment boundary. Begin fixture development with #4233; add and run the relevant tests as #4234, #4235, and #4236 are implemented.
- Dominant language
- Rust
- Stars
- 4.9k
- Forks
- 442
- Avg merge
- 2d 23m
- Merged PRs (30d)
- 153
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/iggy
-
server test
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
connectors: runtime discards the sink consume FFI status, so failed batches are counted as processed Openbug connectors rust
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
bug python sdk
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
kwakseongjae/auto-hwp#319 ·
-
area:cli bug filter-quality good first issue priority:medium
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
bevyengine/bevy#25861 ·
-
comp-datalake
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
ClickHouse/ClickHouse#121222 ·
-
enhancement remote
Difficulty 2/5 1-3 hours Newbie friendliness 68/100