Why no support for range partitioning in USQL?
- Dominant language
- C#
- Stars
- 231
- Forks
- 666
- PR merge metrics
- No merged PRs in 30d
Description
As a result of this limitation, I have to create distinct partitions for all the dates plus it does not support high precision datetime datatypes. I ran into a similar limitation with HIVEQL on HDINSIGHT clusters.
If you use the same column for partitioning, it overwrites the column data.
I have a 700 million row table which results in 3423 distinct partitions.
Trying to create these partitions @pdate1.......@pdate3423 results in a gigantic script.
The USQL job is stuck on "preparing" phase for 3 hrs and then quits without doing anything.
SEVERITY Info
DESCRIPTION Job was cancelled by Yarn.
MESSAGE Job was cancelled.
result:CompilationFailed
Contributor guide
Assessment
This issue has not been assessed yet.