alibaba / alibaba/DataX

基于规则的转换组件(RBT)

Open
#1,410 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

datax 注意力大部分是reader/writer, datax的设计核心在于星型结构,即,reader/writer可以随意撤换,中间exchanger作适配,Transfomer应该是最重要的角色,但datax本身Transfomer比较简单,本组使用xql/规则实现Transfomer构建,适配reader/writer,目前以图库为writer,但可作为探索,实现低代码datax
![设置转换规则](https://user-images.githubusercontent.com/721472/174548658-c7eef48c-d79a-4cf1-8bf0-74daa418489a.png)
其中,xql 数据库之sql;elasticserch esdel等
![RBT原理](https://user-images.githubusercontent.com/721472/174548695-429b7a71-17b1-4614-8d32-11aa43b27146.png)

详细的设计原理 https://www.toutiao.com/article/7109709746328060431/

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing DataX's existing reader, writer, exchanger, and Transformer design, then read the linked RBT design article and the diagrams in the issue. The scope is complete when a rule-based Transformer design is defined and its compatibility with interchangeable readers and writers is demonstrated, including the described graph-database writer scenario.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, java, sql
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.