[Bug] [milvus] sink database config is no use
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 204
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
Seatunnel Milvus source and Milvus sink, I has config:
{
"env": {
"execution.parallelism": 3,
"job.mode": "BATCH"
},
"source": [
{
"plugin_name": "Milvus",
"url": "http://10.18.x.x:9091",
"token": "xxx:xxx",
"database": "default",
"collection": "knowledge_embeddings_totals"
}
],
"sink": [
{
"plugin_name": "Milvus",
"url": "http://x.com:9091",
"token": "yyy:yyy",
"database": "zikang_migrate",
"enable_upsert": "true"
}
]
}
Got error bellow:
PERMISSION_DENIED: PrivilegeCreateCollection: permission deny to yyy in the `default` database
The database configuration under the sink section doesn't seem to take effect — the sink still tries to write to the default database, which causes an error because the user doesn't have write permissions on the default database.
After I granted the user yyy write permissions on the default database, I checked the job status and found the following:
vertexInfoMap":[{"vertexId":1,"type":"source","vertexName":"pipeline-1 [Source[0]-Milvus]","tablePaths":["default.knowledge_embeddings_totals"]},{"vertexId":2,"type":"sink","vertexName":"pipeline-1 [Sink[0]-Milvus-default.knowledge_embeddings_totals]","tablePaths":["zikang_migrate.knowledge_embeddings_totals"]
Sink[0]-Milvus-default.knowledge_embeddings_totals] , Ideally, it should have been zikang_migrate.knowledge_embeddings_totals
### SeaTunnel Version
2.3.9
### SeaTunnel Config
```conf
{
"env": {
"execution.parallelism": 3,
"job.mode": "BATCH"
},
"source": [
{
"plugin_name": "Milvus",
"url": "http://10.18.x.x:9091",
"token": "xxx:xxx",
"database": "default",
"collection": "knowledge_embeddings_totals"
}
],
"sink": [
{
"plugin_name": "Milvus",
"url": "http://x.com:9091",
"token": "yyy:yyy",
"database": "zikang_migrate",
"enable_upsert": "true"
}
]
}
```
### Running Command
```shell
curl -X POST "http://10.18.xx.xx:10901/submit-job" \
-H "Content-Type: multipart/form-data" --data-binary @/home/seatunnel/m4.conf
```
### Error Exception
```log
PERMISSION_DENIED: PrivilegeCreateCollection: permission deny to yyy in the `default` database
```
### Zeta or Flink or Spark Version
_No response_
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start from the Milvus sink configuration used by the submitted m4.conf through the /submit-job entry point, and trace how the sink database is selected when creating the target collection. Reproduce the permission error with the provided source and sink settings, then verify that the sink uses zikang_migrate rather than default.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100