alibaba / alibaba/DataX

datax不能读取我的分区字段 dt

Open
#568 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

我的hive表A ,分区字段为dt 日期格式。
目前需要把hive的数据导入到mysql,并且在mysql表里保存hive表的dt字段。
sqoop会把分区字段当成一个字段对待,而datax不会把dt分区当成字段。
所以,目前的问题是 datax不能读取到我的分区字段
各位大佬有什么好的建议吗?
谢谢

例如hive表A有如下字段(name,age,sex),并且使用dt分区。sqoop抽取数据会读出4个字段,而datax只会读取name,age,sex

Contributor guide

No contributing guide indexed for this repository

Research direction

No source file or test is named. Start by locating the DataX Hive reader and examining how partition fields are handled, then review the MySQL destination path. Done means the Hive dt partition is available as a field during the transfer and is preserved in MySQL.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.