splitPk后,生成重复数据文件
Open
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
读插件为oceanbasev10reader,写插件为txtfilewriter
不加splitPk参数,仅生成一个文件。加上splitPk后,生成了多个文件,但是核查发现,每个文件的内容都一模一样,都是全量数据。
确认了splitPk的字段是递增且唯一的,请问该如何保证生成的多个文件应该为实际拆分任务查询出来的数据,而不是全量数据呢?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the case with oceanbasev10reader, txtfilewriter, and splitPk, comparing the generated files with and without splitPk. Trace how each split task builds and executes its query; done means multiple output files contain their respective task rows rather than identical full datasets.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100