alibaba / alibaba/DataX

postgresqlwriter在写入表的枚举字段时,提示ERROR: type "userstatus" does not exist

Open
#1,733 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

场景:在用postgresqlreader从某个数据库的用户表读取数据,然后使用postgresqlwriter向另一个数据库的用户表写入数据,两张用户表的状态(status)字段在postgres中是一个自定义的枚举类型UserStatus,写入数据时报错如下:

`2023-03-29 09:35:59.652 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select id, name, username, 'ACTIVE' as status, created_at, updated_at, org_id from public.auth_user
] jdbcUrl:[jdbc:postgresql://localhost:5432/ma_dev].
2023-03-29 09:35:59.684 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select id, name, username, 'ACTIVE' as status, created_at, updated_at, org_id from public.auth_user
] jdbcUrl:[jdbc:postgresql://localhost:5432/ma_dev].
2023-03-29 09:35:59.719 [0-0-0-writer] WARN CommonRdbmsWriter$Task - 回滚此次写入, 采用每次写入一行方式提交. 因为:Batch entry 0 INSERT INTO public.auth_user (id,name,username,status,created_at,updated_at,org_id) VALUES('clbx2xvfa0004amq0ezyuovfx'::text,'管理员'::varchar,'admin'::varchar,'ACTIVE'::UserStatus,'2022-12-30 19:12:55.127+08'::timestamptz,'2022-12-30 19:12:55.127+08'::timestamptz,'cle3xb41u0000tdjsz1zd519f'::text) was aborted: ERROR: type "userstatus" does not exist
位置:129 Call getNextException to see other errors in the batch.
2023-03-29 09:35:59.726 [0-0-0-writer] ERROR StdoutPluginCollector -
org.postgresql.util.PSQLException: ERROR: type "userstatus" does not exist
位置:129
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2675) ~[postgresql-42.3.3.jar:42.3.3]
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2365) ~[postgresql-42.3.3.jar:42.3.3]
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:355) ~[postgresql-42.3.3.jar:42.3.3]
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:490) ~[postgresql-42.3.3.jar:42.3.3]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:408) ~[postgresql-42.3.3.jar:42.3.3]
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:166) ~[postgresql-42.3.3.jar:42.3.3]
at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:155) ~[postgresql-42.3.3.jar:42.3.3]
at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.doOneInsert(CommonRdbmsWriter.java:382) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.doBatchInsert(CommonRdbmsWriter.java:362) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:297) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:319) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.postgresqlwriter.PostgresqlWriter$Task.startWrite(PostgresqlWriter.java:87) ~[postgresqlwriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
2023-03-29 09:35:59.729 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据:
{"exception":"ERROR: type \"userstatus\" does not exist\n 位置:129","record":[{"byteSize":25,"index":0,"rawData":"clbx2xvfa0004amq0ezyuovfx","type":"STRING"},{"byteSize":3,"index":1,"rawData":"管理员","type":"STRING"},{"byteSize":5,"index":2,"rawData":"admin","type":"STRING"},{"byteSize":6,"index":3,"rawData":"ACTIVE","type":"STRING"},{"byteSize":8,"index":4,"rawData":1672398775127,"type":"DATE"},{"byteSize":8,"index":5,"rawData":1672398775127,"type":"DATE"},{"byteSize":25,"index":6,"rawData":"cle3xb41u0000tdjsz1zd519f","type":"STRING"}],"type":"writer"}
2023-03-29 09:35:59.731 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据:
{"exception":"ERROR: type \"userstatus\" does not exist\n 位置:129","record":[{"byteSize":25,"index":0,"rawData":"clbx2xvfa0004amq0ezyuovfi","type":"STRING"},{"byteSize":4,"index":1,"rawData":"管理员1","type":"STRING"},{"byteSize":6,"index":2,"rawData":"admin1","type":"STRING"},{"byteSize":6,"index":3,"rawData":"ACTIVE","type":"STRING"},{"byteSize":8,"index":4,"rawData":1672398775127,"type":"DATE"},{"byteSize":8,"index":5,"rawData":1672398775127,"type":"DATE"},{"byteSize":25,"index":6,"rawData":"cle3xb41u0000tdjsz1zd519f","type":"STRING"}],"type":"writer"}`

请问这个枚举字段如何转换?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with CommonRdbmsWriter.java at Task.doOneInsert and trace how PostgresqlWriter.java prepares the INSERT shown in the stack trace. Reproduce the PostgreSQL enum-column job and inspect the PostgreSQL JDBC binding or cast involved. Done means the same UserStatus values can be written to the destination table without the missing-type error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, postgresql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.