apache / apache/hudi

aws query engine don't support all hudi promotion type

Open
#16,644 1 comment 0 reactions 1 assignee Claimed by @yihua View on GitHub
from-jira priority:high type:improvement
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

athena and spectrum have limitation in promotion type support. Then hudi has better to rollback the commit when the sync operation failed. OTW the table would be in a corrupted state for the query engine.

 

|          | int->bigint | int->float | float->double | int->string | 
|----------|-------------|------------|---------------|-------------|
| spark    | yes         | yes        | yes           | yes         |
| athena   | yes         | yes        | yes           | no          |
| spectrum | yes         | no         | yes           | no          |

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-8292
- Type: Improvement
- Fix version(s):
- 1.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.