duckdb / duckdb/duckdb-spatial

Support appending multiple layers to GeoPackage (GPKG) in COPY TO

Open
#868 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
708
Forks
96
Avg merge
1d 21h
Merged PRs (30d)
5

Description

### Summary
Currently, exporting to GeoPackage (`FORMAT GDAL, DRIVER 'GPKG'`) overwrites the target file on every `COPY ... TO` execution (or fails due to file locks on temporary files).

We request support for an `APPEND` / `UPDATE` mode when writing to multi-layer formats like GeoPackage, allowing users to sequentially add multiple layers into the same `.gpkg` file.

### Motivation & Use Case
The OGC GeoPackage format is fundamentally an SQLite container explicitly designed to store multiple spatial and non-spatial tables (layers) within a single self-contained file.

In modern GIS workflows:
- **GeoPackage has largely replaced the legacy Shapefile format**, specifically to avoid dealing with dozens of loose files (like `.shp`, `.dbf`, `.shx`, `.prj`) for related layers of a single project.
- Users frequently perform multiple analytical queries, spatial joins, or aggregations in DuckDB and want to bundle the resulting layers (e.g., `polygons_layer`, `points_layer`, `metadata_table`) into a **single GeoPackage** for delivery or use in QGIS/ArcGIS.
- Overwriting the entire `.gpkg` container per `COPY` statement strips GeoPackage of one of its most important core capabilities.

### Proposed Syntax / Behavior

Introduce an option in the `COPY ... WITH (...)` clause (such as `APPEND TRUE` or `MODE 'APPEND' / 'UPDATE'`):

```sql
-- 1. Create file with the first layer
COPY (
SELECT * FROM administrative_boundaries
) TO 'project_data.gpkg'
WITH (
FORMAT GDAL,
DRIVER 'GPKG',
LAYER_NAME 'boundaries'
);

-- 2. Append a second layer to the existing GeoPackage
COPY (
SELECT * FROM points_of_interest
) TO 'project_data.gpkg'
WITH (
FORMAT GDAL,
DRIVER 'GPKG',
LAYER_NAME 'poi',
APPEND TRUE -- or: MODE 'APPEND'
);
```

### Technical Note / GDAL Feasibility

Under the hood, GDAL's GPKG driver natively supports this via:

- Opening an existing dataset with write access (GDALOpenEx(..., GDAL_OF_UPDATE | GDAL_OF_VECTOR)).

- Calling GDALDataset::CreateLayer() on the opened dataset instead of invoking GDALCreate() from scratch.

### Current Workarounds

Currently, users are forced to leave DuckDB and use external tools like ogr2ogr -update -append or Python (pyogrio / geopandas with mode="a"). Supporting this natively in DuckDB Spatial would greatly streamline end-to-end SQL GIS workflows.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the COPY ... WITH (...) GDAL path and compare the current GDALCreate() flow with the mentioned GDALOpenEx(... GDAL_OF_UPDATE | GDAL_OF_VECTOR) and GDALDataset::CreateLayer() entry points. Define and test the chosen APPEND or UPDATE option so sequential COPY statements create distinct layers in one GeoPackage without overwriting existing layers.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.