citusdata / citusdata/cstore_fdw

Setup for analytics data storage

Open
#185 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
1.8k
Forks
169
PR merge metrics
No merged PRs in 30d

Description

Hi thank you for your great work at citusdata! We just setup a server with an cstore_fdw installation.
The initial copy of our table (around 80 milion data rows) into the foreign table took about 20 minutes. The query performance boost with the given approach is amazing!
The data in the origin table changes very rapidly (inserts and updates). While for our day to day analysis (which we plan to use the cstore for) 24 hour old data would be fresh enough i am not sure how the best practice for setting up a analytics only table with cstore should look like.

My current thougts are:
1. setup a replica while the master has only a cstore installed (with an empty table) the replica shall fill the data in batches every 24 hours.
1.1. cause the initial fill of the table takes more than the max_standby_streaming_delay this seems to be kind of problematic
2. like 1 but with triggers on the slave that would insert/update/delete records in the cstore
2.2. is not possible since cstore does only support insert

The setup with the trigger looks like the most up to date version and would take much less resources (assuming upserting/deleting would be not that expansive as a setup from scratch.

Is there any input you can give me about best practices here or can you give me an update about the upsert/deletion functionality of cstore_fdw (discussed also in #2 #175 and last but not least in #118)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.