duckdb / duckdb/duckdb-postgres

Parallel COPY to Postgres ?

Open
#185 3 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
372
Forks
105
Avg merge
10h 19m
Merged PRs (30d)
18

Description

### What happens?

Hello,
I tried to do a parallel import in Postgres.
I wonder if postgres_scanner allows this ?

### To Reproduce

I tried :
```
ATTACH 'user=postgres host=/var/run/postgresql port=5440' AS db (TYPE postgres);
COPY db.votes2 FROM 'data.zstd/*' (FORMAT parquet);
```
I tried to change file format (csv, parquet), compression, multiples files...
I also tried to first import in duckdb, then, copy to Postgres.
I only see one COPY process on Postgres.

### OS:

Linux

### PostgreSQL Version:

16

### DuckDB Version:

v0.10.0 20b1486d11

### DuckDB Client:

CLI

### Full Name:

Adrien Nayrat

### Affiliation:

Self Employed

### Have you tried this on the latest `main` branch?

- [X] I agree

### Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

- [X] I agree

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.