citusdata / citusdata/citus_docs

Evaluate updating Bulk Loading section's Note

Open
#241 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
CSS
Stars
58
Forks
60
PR merge metrics
No merged PRs in 30d

Description

We have the following note as part of the bulk loading section:

"There is no notion of snapshot isolation across shards, which means that a multi-shard SELECT that runs concurrently with a COPY might see it committed on some shards, but not on others. If the user is storing events data, he may occasionally observe small gaps in recent data. It is up to applications to deal with this if it is a problem (e.g. exclude the most recent data from queries, or use some lock).

If COPY fails to open a connection for a shard placement then it behaves in the same way as INSERT, namely to mark the placement(s) as inactive unless there are no more active placements. If any other failure occurs after connecting, the transaction is rolled back and thus no metadata changes are made."

I had three questions on this section:

1. When the user runs ``COPY``, Citus currently uses transactions to commit or rollback batch loading of data. I ran multiple ``COPY`` operations and concurrent ``SELECT count(*) FROM github_events;`` and I saw transactional behavior here. Are we worried about the window where parallel commits across machines take time to complete? -- Isn't that a small window?
2. The first and second paragraphs in this note seem unrelated. Do we have two notes?
3. Do we want to document ``\COPY`` or ``COPY``? [PostgreSQL's documentation](https://www.postgresql.org/docs/9.6/static/tutorial-populate.html) generally talks about ``COPY``. That said, ``\COPY`` is more convenient to use.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.