alibaba / alibaba/DataX

split的优化点

Open
#2,157 3 comments 2 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

1.IS NOT NULL应该支持配置项
![image](https://github.com/user-attachments/assets/e7faeca3-b9c5-46d7-b0b6-4ec464d1424e)
这个IS NOT NULL的拼接调整为配置可选会更好,因为对于mysql本身就是忽略null,但是在min和max加了IS NOT NULL会导致整颗索引树扫描
![image](https://github.com/user-attachments/assets/352692f6-de2e-447d-864c-fd2d0f4ef96e)

2.做分库分表迁移,存在自增id和雪花算法,使用默认split会有数据倾斜,如果split支持按自己配置的切分段执行会更好

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by locating the split implementation and its configuration and test coverage, then clarify whether the requested work is configurable IS NOT NULL handling, custom split ranges, or both. Done should include an agreed configuration design and tests covering MySQL null handling and migrations with mixed ID strategies.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.