citusdata / citusdata/citus

Add multi-column partition keys.

Open
#1,345 0 comments 3 reactions 0 assignees View on GitHub
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

After moving to our partition pruning logic, its relatively straightforward to perform pruning of multiple columns. That'd allow to use more complex partitioning keys more easily than it's currently possible using composite keys, while also having lower runtime & storage overhead.

The code-changes individually are all not that complicated, it's the user interface that's a bit harder. We have several functions that are written in a way that's not directly conducive to using more complex partitioning keys:
* `create_distributed_table()` - `distributionColumnName` is a single text column, could be overloaded with variant expecting an array?
* `master_get_table_metadata()` - returns partition key as expression, could reasonably be expected to just return a more complex expression, but would probably still break some things
* `get_shard_id_for_distribution_column()` - expects a single argument for key, but could be expanded to be a vararg function

(there's probably more, these are just the ones that came to mind)

There's also `pg_dist_shard` shardminvalue/maxvalue which currently stores the key as text. I guess we could store it as an array or such instead?

Note: This is not multi-level partitioning. But some though for that is probably appropriate, to avoid having to repeatedly change the user-interface in non-compatible ways.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.