alibaba / alibaba/DataX

mongodbreader may should not always use `MULTIPLE` mode

Open
#1,645 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

when mongodbreader connect to db server it use :
`public MongoClient(List seeds, List credentialsList)`
construction method to instance MongoClient.
this will cause client use MULTIPLE mode to make up cluster settings.
`Cluster created with settings {hosts=[10.7.x.x:30016], mode=MULTIPLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}`

In my scenario,mongodb was run on k8s with REPLICA_SET mode, this will cause client keep trying to touch backend service:
```
Server 10.7.x.x:30016 is no longer a member of the replica set. Removing from client view of cluster.
Canonical address mongodb-1.mongodb-headless.bangwo8-db-prod.svc.cluster.local:27017 does not match server address. Removing 10.7.x.x:30016 from client view of cluster
Updating cluster description to {type=REPLICA_SET, servers=[{address=mongodb-0.mongodb-headless.bangwo8-db-prod.svc.cluster.local:27017, type=UNKNOWN, state=CONNECTING}, {address=mongodb-arbiter-0.mongodb-arbiter-headless.bangwo8-db-prod.svc.cluster.local:27017, type=UNKNOWN, state=CONNECTING}, {address=mongodb-1.mongodb-headless.bangwo8-db-prod.svc.cluster.local:27017, type=UNKNOWN, state=CONNECTING}]
```
finally datax job will faild with java.net.UnknownHostException.

but if I change to use construction method :
`public MongoClient(ServerAddress addr, List credentialsList)`
the MongoClient will use SINGLE to make up cluster settings. And then I can search data normally.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the mongodbreader code that constructs MongoClient and compare the two constructors shown in the issue, then reproduce the connection against a replica set using the reported server configuration. Done means the reader connects without repeatedly removing replica-set members or failing with UnknownHostException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mongodb
Domain
database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.