apache / apache/iggy

connectors: sink start leaks the plugin instance when consumer setup fails

Open
#4,101 0 comments 0 reactions 1 assignee Claimed by @mlevkov View on GitHub
bug connectors rust
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

## What happens

`SinkManager::start_connector` in `core/connectors/runtime/src/manager/sink.rs` records the new
plugin id on `SinkDetails` only after `setup_sink_consumers` has returned. A failure between
`init_sink` and that record leaves the plugin holding an instance whose id nothing outside it knows,
and `stop_connector` closes whatever `details.info.id` still holds, which is the previous instance.

This is the same window #4062 described on the source path, one manager over.

## Why it is narrower than the source case

Sink consume tasks exit when the `watch::Sender` in `SinkDetails` drops, so the runtime half of the
leak cleans itself up. Only the FFI half is stranded: the plugin keeps an open instance that nothing
will close for the life of the process.

That is why this needs the guard and not the second half of the #4064 fix. The source path also had
to move the id record under the same lock as the task spawn, because `SOURCE_SENDERS` is a process
global that no drop can reach. The sink has no equivalent.

## Fix

Use the same `SourceInstanceGuard` shape #4064 added, armed once `init_sink` succeeds and disarmed
once the id is recorded on `SinkDetails`.

## Provenance

Raised by @hubcio in review on #4064, out of that PR's diff and explicitly non-blocking for it.
Filed separately so it does not disappear when #4064 merges.

### Contribution

- [x] I'm willing to submit a pull request to fix this bug

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.