Azure / Azure/usql

Why no support for range partitioning in USQL?

Open
#111 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
231
Forks
666
PR merge metrics
No merged PRs in 30d

Description

As a result of this limitation, I have to create distinct partitions for all the dates plus it does not support high precision datetime datatypes. I ran into a similar limitation with HIVEQL on HDINSIGHT clusters.
If you use the same column for partitioning, it overwrites the column data.

I have a 700 million row table which results in 3423 distinct partitions.
Trying to create these partitions @pdate1.......@pdate3423 results in a gigantic script.

The USQL job is stuck on "preparing" phase for 3 hrs and then quits without doing anything.
SEVERITY Info
DESCRIPTION Job was cancelled by Yarn.
MESSAGE Job was cancelled.
result:CompilationFailed

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.