alibaba / alibaba/DataX

使用Postgresqlreader读取蟑螂db分页不支持

Open
#406 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-06], Description:[执行数据库 Sql 失败, 请检查您的配置的 column/table/where/querySql或者向 DBA 寻求帮助.]. - 执行的SQL为: select user_id,product_id,product_type,secu_code,mkt_type,secu_type,side,price,quantity,prequantity,commission,stamp_tax,trans_fee,status,seq,order_seq,share_count,trade_position,initial_position,create_time,update_time from trade 具体错误信息为:org.postgresql.util.PSQLException: ERROR: unimplemented: execute row count limits not supported: 1000 of 99773
Hint: See: https://github.com/cockroachdb/cockroach/issues/4035
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:26)
at com.alibaba.datax.plugin.rdbms.util.RdbmsException.asQueryException(RdbmsException.java:97)
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Task.startRead(CommonRdbmsReader.java:220)
at com.alibaba.datax.plugin.reader.postgresqlreader.PostgresqlReader$Task.startRead(PostgresqlReader.java:70)
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57)
at java.lang.Thread.run(Thread.java:748)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with PostgresqlReader$Task.startRead in PostgresqlReader.java and CommonRdbmsReader$Task.startRead in CommonRdbmsReader.java, then reproduce the reported query against CockroachDB using the provided configuration. Determine how the reader applies row limits and verify that pagination works without triggering the reported error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, postgresql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.