apache / apache/seatunnel

[Bug] [Transform DefineSinkType] DefineSinkType Not Found

Open
#9,702 3 comments 0 reactions 0 assignees View on GitHub
bug fixed-in-dev
Dominant language
Java
Stars
9.7k
Forks
2.4k
Avg merge
3d 13h
Merged PRs (30d)
203

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.

### What happened

The main method caused an error: Plugin PluginIdentifier{engineType='seatunnel', pluginType='transform', pluginName='**DefineSinkType**'} **not found**

here is my js code

```javascript
{
plugin_name: 'DefineSinkType',
columns: [
{
column: "db_source_from",
type: "BIT(8)"
}
]
}
```

### SeaTunnel Version

2.3.11

### SeaTunnel Config

```conf
{
"env": {
"job.mode": "BATCH"
},
"source": [
{
"plugin_name": "Jdbc",
"url": "jdbc:mysql://localhost:3306/local_wms_art?allowMultiQueries=true&nullCatalogMeansCurrent=true&useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=GMT%2b8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true&rewriteBatchedStatements=true",
"driver": "com.mysql.cj.jdbc.Driver",
"connection_check_timeout_sec": 30,
"table_list": [
{
"table_path": "local_wms_art.orders",
"query": "SELECT *, 1 AS db_source_from FROM local_wms_art.orders"
},
{
"table_path": "local_wms_art.info_ccm",
"query": "SELECT *, 1 AS db_source_from FROM local_wms_art.info_ccm"
},
{
"table_path": "local_wms_art._merge_test_table1",
"query": "SELECT *, 1 AS db_source_from FROM local_wms_art._merge_test_table1"
}
],
"plugin_output": "a"
},
{
"plugin_name": "Jdbc",
"url": "jdbc:mysql://localhost:3306/local_wms?allowMultiQueries=true&nullCatalogMeansCurrent=true&useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=GMT%2b8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true&rewriteBatchedStatements=true",
"driver": "com.mysql.cj.jdbc.Driver",
"connection_check_timeout_sec": 30,
"table_list": [
{
"table_path": "local_wms.orders",
"query": "SELECT *, 2 AS db_source_from FROM local_wms.orders"
},
{
"table_path": "local_wms.info_ccm",
"query": "SELECT *, 2 AS db_source_from FROM local_wms.info_ccm"
},
{
"table_path": "local_wms._merge_test_table1",
"query": "SELECT *, 2 AS db_source_from FROM local_wms._merge_test_table1"
}
],
"plugin_output": "b"
}
],
"transform": [
{
"plugin_name": "DefineSinkType",
"columns": [
{
"column": "db_source_from",
"type": "BIT(8)"
}
]
}
],
"sink": [
{
"plugin_name": "Jdbc",
"url": "jdbc:mysql://localhost:3306?allowMultiQueries=true&nullCatalogMeansCurrent=true&useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=GMT%2b8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true&rewriteBatchedStatements=true",
"driver": "com.mysql.cj.jdbc.Driver",
"connection_check_timeout_sec": 30,
"schema_save_mode": "RECREATE_SCHEMA",
"database": "merge_wms",
"generate_sink_sql": true,
"plugin_input": "a"
},
{
"plugin_name": "Jdbc",
"url": "jdbc:mysql://localhost:3306?allowMultiQueries=true&nullCatalogMeansCurrent=true&useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=GMT%2b8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true&rewriteBatchedStatements=true",
"driver": "com.mysql.cj.jdbc.Driver",
"connection_check_timeout_sec": 30,
"schema_save_mode": "RECREATE_SCHEMA",
"database": "merge_wms",
"generate_sink_sql": true,
"plugin_input": "b"
}
]
}
```

### Running Command

```shell
../bin/start-seatunnel-flink-15-connector-v2.sh --config ../config/wms.batch.json
```

### Error Exception

```log
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Plugin PluginIdentifier{engineType='seatunnel', pluginType='transform', pluginName='DefineSinkType'} not found.
```

### Zeta or Flink or Spark Version

Flink 1.18.1

### Java or Scala Version

_No response_

### Screenshots

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the supplied config/wms.batch.json and the start-seatunnel-flink-15-connector-v2.sh command, then check how the DefineSinkType transform is registered or packaged in SeaTunnel 2.3.11. Reproduce the missing-plugin error and determine whether the transform should be available; done means the supported configuration or the required plugin availability is clearly identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
data-engineering, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.