citusdata / citusdata/cstore_fdw

Use PG internal file (relfilenode/page) structure to store cstore data

Open
#138 4 comments 0 reactions 0 assignees View on GitHub
fixed-in-development-version
Dominant language
C
Stars
1.8k
Forks
169
PR merge metrics
No merged PRs in 30d

Description

Cstore_fdw stores its meta-data and data files on the file system in its own folder. This has becoming a problem for cstore. If we can start using internal file structure we can
- drop superuser requirement since we will be staying inside pg landscape
- support pg_basebackup
- support streaming replication via WAL logging
- survive PG version upgrade without data loss or corruption.

There is a prototype available at branch [store_data_in_internal_storage](https://github.com/citusdata/cstore_fdw/tree/store_data_in_internal_storage) . It needs some refinement.

We may need to implement a migration UDF to move data stored in file system to internal storage.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.