citusdata / citusdata/citus

Columnar storage and small stripes

Open
#7,401 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

I am evaluating Citus Columnar to replace our Sql Server Clustered Columnstore indexes and I came upon stripes which are essentially the equivalent to segment in Sql Server. Based on the documentation, stripes needs to be as much large as possible (default 150,000) considering the fact how columnstore engines work to offer max performance. Our use case involves real time analytics (every 1 minute we do inserts) but we see an issue the way columnar works in Citus. Every minute we have between 5-1000 new rows. That creates a problem because based on the Citus documentation, tiny stripes will mess table.
In Sql Server this is avoided with the use of 'Delta rowgroup' which essentially works as a `buffer` until the number of rows reach 1,048,576 (number of rows on each segment) and then they are added and sealed in a segment.
Is this functionality something that is planned for Citus Columnar?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.