apache / apache/pulsar

Sending "stop" to a connector should be idempotent.

Open
#18,119 1 comment 0 reactions 0 assignees View on GitHub
Stale
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

We're running pulsar in Kubernetes and ran into a situation where the "stop" command was called on a connector (a sink in this case) but the kubernetes statefulset was not removed. Calling "stop" again on the same connector returned an HTTP 400 error with "operation not permitted" because pulsar thought that the connector was already stopped.

I was able to work around this by first calling "start" on the connector, either from the pulsar-admin CLI or the rest endpoint, and then calling "stop" again. This time the stateful set was removed.

### Solution

Instead of returning different responses (HTTP 200 vs. 400) depending on the state of the connector, I think it would be better if a "stop" call on an already stopped connector would try again to stop the connector and clean up any hanging resources, and then return a 200 if everything went ok. So several calls to stop on a connector in a row should all return HTTP 200 assuming that the connector was stopped successfully.

### Alternatives

_No response_

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with the connector stop handling exposed through the pulsar-admin CLI and REST endpoint, focusing on sink connectors and cleanup of the Kubernetes StatefulSet. Reproduce repeated stop calls and verify that an already stopped connector retries cleanup, returns HTTP 200, and leaves no hanging resources.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kubernetes
Domain
api, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.