duckdb / duckdb/duckdb-spatial

Avoid scanning entire Geodatabase file

Open
#68 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
708
Forks
96
Avg merge
1d 21h
Merged PRs (30d)
5

Description

It appears an entire scan of a Geodatabase file is needed before a single record can be returned. The following pulls one record from an 870 MB file and takes ~36 seconds to do so.

```sql
select FIPS_CODE FROM st_read('SF_Professional_AK.gdb/a00000011.gdbtable') LIMIT 1;
```

```
┌───────────┐
│ FIPS_CODE │
│ varchar │
├───────────┤
│ 11111 │
└───────────┘
```

```
Run Time (s): real 36.234 user 33.853635 sys 2.386451
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the SQL query against SF_Professional_AK.gdb/a00000011.gdbtable and trace the st_read entry point in the C extension. Compare the work performed for LIMIT 1 with the reported 36-second full-file scan; done means a single record can be returned without scanning the entire 870 MB file.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, sql
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.