apache / apache/seatunnel

[Bug] [UDF] upload udf lib,run the job,error

Open
#10,403 4 comments 0 reactions 0 assignees View on GitHub
bug help wanted
Dominant language
Java
Stars
9.7k
Forks
2.4k
Avg merge
3d 13h
Merged PRs (30d)
203

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.

### What happened

Image

Image

According to https://seatunnel.apache.org/zh-CN/docs/2.3.10/transform-v2/sql-udf/, I wrote a UDF and placed it in the lib directory. When running, it prompts that the class java.lang.ClassNotFoundException: org.apache.seatunnel.api.common.CommonOptions cannot be found. Moreover, I checked that the documentation requires the seatunnel-api version introduced in the UDF project to be 2.3.2, but in this version, CommonOptions still uses result_table_name and source_table_name, which conflicts with plugin_output/plugin_input in subsequent versions.

### SeaTunnel Version

2.3.10

### SeaTunnel Config

```conf
env{
job.name="future_pos"
parallelism = 1
job.mode = "BATCH"
}

source {
Jdbc {
url = "jdbc:oracle:thin:@192.168.98.xx:1521:xxx"
driver = "oracle.jdbc.OracleDriver"
connection_check_timeout_sec = 100
user = "dbusrposnew"
password = "futureposnew"
split.size = 10000
table_list = [
{
table_path = "DBUSRPOSNEW.SALEHEAD"
# Use query filetr rows & columns
query = "select * from DBUSRPOSNEW.SALEHEAD where RQSJ>=TO_DATE('${this_day}', 'YYYY/MM/DD HH24:MI:SS')"
}

]
}
}

transform {
Sql {
plugin_output="t"
plugin_input="t1"
query = """
select BILLNO, MKT, SYJH, FPHM, DJLB, BC, RQSJ,
TransformsGBK(SYYH) SYYH , HYKH, JFKH, THSQ, GHSQ, HYSQ, SQKH, SQKTYPE, SQKZKFD, YSJE, SJFK, ZL, SSWR_SYSY, FK_SYSY, HJZJE, HJZSL, HJZKE, HYZKE, YHZKE, LSZKE, BUYERINFO, JDFHDD, SALEFPHM, LJJF, BCJF, MEMO, STR1, TransformsGBK(STR2) STR2, STR3, STR4, STR5, NUM1, NUM2, NUM3, NUM4, NUM5, SENDRQSJ, STATUS, CUSTTYPE, HHFLAG, MANAUNIT, FQFLAG, JFFLAG, SAFLAG, FQJE, YHZSZK, YBILLNO, YSYJH, YFPHM, CRMBILLNO, STR8, CHANNEL, HYCID, HYTYPE FROM t
"""
}
}

sink {
Doris {
fenodes = "192.168.0.123:8030,192.168.0.456:8030,192.168.0.789:8030"
username = root
password = "root"
database = "ods_future_pos"
table = "ods_${table_name}"

sink.label-prefix = "future-pos_${table_name}-jdbc"
sink.enable-2pc = "true"
sink.enable-delete = "true"
save_mode_create_template="CREATE TABLE IF NOT EXISTS `${database}`.`${table_name}` (${rowtype_primary_key},${rowtype_fields},dt datetime(0) default current_timestamp(0) on update current_timestamp(0)) ENGINE=OLAP UNIQUE KEY (${rowtype_primary_key}) DISTRIBUTED BY HASH (${rowtype_primary_key}) BUCKETS 3 PROPERTIES ( \"replication_allocation\" = \"tag.location.default: 3\",\"in_memory\" = \"false\",\"storage_format\" = \"V2\", \"disable_auto_compaction\" = \"false\")"
doris.config {
format = "json"
read_json_by_line = "true"
}
}
}
```

### Running Command

### Error Exception

```log
java.lang.ClassNotFoundException: org.apache.seatunnel.api.common.CommonOptions
```

### Zeta or Flink or Spark Version

zeta

### Java or Scala Version

1.8

### Screenshots

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the UDF documentation at the linked SQL UDF page and the UDF project's seatunnel-api dependency, then inspect how org.apache.seatunnel.api.common.CommonOptions is packaged or loaded from the lib directory in SeaTunnel 2.3.10. Reproduce the ClassNotFoundException with the supplied configuration and verify that the documented API version and plugin_input/plugin_output naming agree with the release.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.