hive外部表导出到mysql串行
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
背景是这样的
1.在hive上面依据其他的表生成一个表hdfs文件夹(temp_table),以parquet格式存储,字段顺序为b,c,d,e,a
2.然后create external table 新建一个外部表(temp_table) 字段顺序是a,b,c,d,e (由于parquet文件字段顺序不对,所以做调整),location 这个hdfs文件夹
3.在hive上select * from temp_table 可以得到正常 a,b,c,d,e这样的顺序结果集,正确的识别了每个字段
但是使用datax ,选择来源类型为hive ,导入到mysql里面的顺序还是b,c,d,e,a这样的顺序,这样就导致了数据串行,没有按照我想要的顺序
请问这个问题是bug嘛?该怎么解决啊
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the Hive source reader's handling of the external table schema and the MySQL destination mapping for the reported DataX job. Reproduce the Parquet column-order case with the described temp_table setup; done means the exported values follow the external table order a,b,c,d,e rather than the underlying file order.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, mysql
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100