apache / apache/seatunnel

[Bug] [milvus] sink database config is no use

Open
#9,678 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
9.7k
Forks
2.4k
Avg merge
3d 9h
Merged PRs (30d)
204

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.

### What happened

Seatunnel Milvus source and Milvus sink, I has config:

{
"env": {
"execution.parallelism": 3,
"job.mode": "BATCH"
},
"source": [
{
"plugin_name": "Milvus",
"url": "http://10.18.x.x:9091",
"token": "xxx:xxx",
"database": "default",
"collection": "knowledge_embeddings_totals"
}
],
"sink": [
{
"plugin_name": "Milvus",
"url": "http://x.com:9091",
"token": "yyy:yyy",
"database": "zikang_migrate",
"enable_upsert": "true"
}
]
}

Got error bellow:
PERMISSION_DENIED: PrivilegeCreateCollection: permission deny to yyy in the `default` database

The database configuration under the sink section doesn't seem to take effect — the sink still tries to write to the default database, which causes an error because the user doesn't have write permissions on the default database.

After I granted the user yyy write permissions on the default database, I checked the job status and found the following:
vertexInfoMap":[{"vertexId":1,"type":"source","vertexName":"pipeline-1 [Source[0]-Milvus]","tablePaths":["default.knowledge_embeddings_totals"]},{"vertexId":2,"type":"sink","vertexName":"pipeline-1 [Sink[0]-Milvus-default.knowledge_embeddings_totals]","tablePaths":["zikang_migrate.knowledge_embeddings_totals"]

Sink[0]-Milvus-default.knowledge_embeddings_totals] , Ideally, it should have been zikang_migrate.knowledge_embeddings_totals

### SeaTunnel Version

2.3.9

### SeaTunnel Config

```conf
{
"env": {
"execution.parallelism": 3,
"job.mode": "BATCH"
},
"source": [
{
"plugin_name": "Milvus",
"url": "http://10.18.x.x:9091",
"token": "xxx:xxx",
"database": "default",
"collection": "knowledge_embeddings_totals"
}
],
"sink": [
{
"plugin_name": "Milvus",
"url": "http://x.com:9091",
"token": "yyy:yyy",
"database": "zikang_migrate",
"enable_upsert": "true"
}
]
}
```

### Running Command

```shell
curl -X POST "http://10.18.xx.xx:10901/submit-job" \
-H "Content-Type: multipart/form-data" --data-binary @/home/seatunnel/m4.conf
```

### Error Exception

```log
PERMISSION_DENIED: PrivilegeCreateCollection: permission deny to yyy in the `default` database
```

### Zeta or Flink or Spark Version

_No response_

### Java or Scala Version

_No response_

### Screenshots

_No response_

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start from the Milvus sink configuration used by the submitted m4.conf through the /submit-job entry point, and trace how the sink database is selected when creating the target collection. Reproduce the permission error with the provided source and sink settings, then verify that the sink uses zikang_migrate rather than default.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.