citusdata / citusdata/cstore_fdw

Parallel COPY FROM crashes server

Open
#214 0 comments 0 reactions 1 assignee Claimed by @pykello View on GitHub
Dominant language
C
Stars
1.8k
Forks
169
PR merge metrics
No merged PRs in 30d

Description

I am trying to load data into 4 different _foreign_ tables. Each `COPY` runs as expected, but whenever I try to parallelize the operation, i.e. run 4 different `COPY` instructions at the same time, the server crashes with the following message:

```
FATAL: terminating connection due to administrator command
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
```

Upon restarting the connection, every time I try to execute a _single_ `COPY` statement, Postgres throws the same error, rendering the tables useless. Only way to have them populated is to drop them, create them again and execute `COPY` on each table, _individually_.

Is there a way to do this process in parallel? Each table has a populating process of their own, there are no concurrent writes to the same table.

The command I run to populate the tables is:

`COPY FROM '' WITH CSV;`

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.