duckdb / duckdb/duckdb-spatial

SHP driver open_options does not seem to work with ENCODING option

Open
#356 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
708
Forks
96
Avg merge
1d 21h
Merged PRs (30d)
5

Description

Example files:
- small - https://opendata.geoportal.gov.pl/prg/adresy/PunktyAdresowe/02/020101.zip
- large - https://integracja.gugik.gov.pl/PRG/pobierz.php?adresy_zbiorcze_shp

SQL example:
```sql
select *
from st_read('/vsizip/./Downloads/020101.zip/PRG_PunktyAdresowe_020101.shp', open_options=['ENCODING=WINDOWS-1250'])
limit 5;
```

Result:
```
┌─────────┬─────────┬─────────┬─────────────┬─────────┬─────────────────────┬─────────┬───────────────────────────────────────────────┐
│ TERYT │ PNA │ SIMC_id │ SIMC_nazwa │ ULIC_id │ ULIC_nazwa │ Numer │ geom │
│ varchar │ varchar │ varchar │ varchar │ varchar │ varchar │ varchar │ geometry │
├─────────┼─────────┼─────────┼─────────────┼─────────┼─────────────────────┼─────────┼───────────────────────────────────────────────┤
│ 020101 │ 59-700 │ 0935989 │ ????INVAL… │ 18648 │ ????INVALID VALUE… │ 34 │ POINT (260060.88199839657 382544.5259835962) │
│ 020101 │ 59-700 │ 0935989 │ ????INVAL… │ 09421 │ ????INVALID VALUE… │ 82 │ POINT (262141.78239842618 381462.537184705) │
│ 020101 │ 59-700 │ 0935989 │ ????INVAL… │ 02058 │ ????INVALID VALUE… │ 13 │ POINT (258755.64489838033 383519.40798285604) │
│ 020101 │ 59-700 │ 0935989 │ ????INVAL… │ 07026 │ ????INVALID VALUE… │ 27C │ POINT (261791.14839842354 382338.35378450714) │
│ 020101 │ 59-700 │ 0935989 │ ????INVAL… │ 07026 │ ????INVALID VALUE… │ 27B │ POINT (261788.7517984234 382329.84118450433) │
└─────────┴─────────┴─────────┴─────────────┴─────────┴─────────────────────┴─────────┴───────────────────────────────────────────────┘
```

Using that encoding in open options works with ogrinfo.

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the SQL example with the small and large SHP ZIP files, using open_options=['ENCODING=WINDOWS-1250'], and compare the output with ogrinfo using the same encoding. The work is done when Polish attribute values are decoded correctly through st_read rather than displayed as invalid characters.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.