apache / apache/pulsar

Suddenly the built in mongo connector stopped the source

Open
#22,777 2 comments 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Read release policy

- [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

### Version

3.2.3

### Minimal reproduce step

{
"configs": {
"mongoUri": "{{mongo_uri}}",
"database": "{{db_name}}",
"collection": "chub_agent",
"syncType": "incr_sync",
"batchSize": "2",
"batchTimeMs": "500"
}
}

### What did you expect to see?

Any changes in the collection will be sourced and pushed to the pulsar topic

### What did you see instead?

We used to source 10 collections. suddenly the connector stops the source from MongoDB. There is no error, it show running but no file is sourced.

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the MongoDB source connector behavior with the reported Pulsar 3.2.3 configuration and inspect connector status and logs while changing the collection. Compare the behavior with the previously working setup; done means collection changes are sourced and published to the Pulsar topic, with a regression test covering the failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mongodb
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.