duckdb / duckdb/duckdb-postgres
ORDER BY col NULLS FIRST ... LIMIT returns only NULL rows through the Postgres scanner
- Dominant language
- C++
- Stars
- 372
- Forks
- 105
- Avg merge
- 10h 19m
- Merged PRs (30d)
- 18
Description
### What happens?
When running a query against a postgres DB with `ORDER BY col NULLS FIRST LIMIT x`, it returns only the null rows.
### To Reproduce
Start a postgres container locally
`docker run -it --rm -p 5432:5432 -e POSTGRES_PASSWORD=password postgres:18`
Then, in a duckdb console the following queries reproduces the issue:
```sql
ATTACH 'host=localhost port=5432 dbname=postgres user=postgres password=password' AS pg (TYPE postgres);
CREATE TABLE pg.example (id text, ts timestamptz);
INSERT INTO pg.example VALUES
('stamped-b', '2026-09-01 00:00:00+00'),
('stamped-a', '2026-09-01 01:00:00+00'),
('fresh-b', NULL),
('fresh-a', NULL);
-- Expected: fresh-a, fresh-b, stamped-b. Actual: fresh-a, fresh-b.
SELECT id FROM pg.example ORDER BY ts NULLS FIRST, id LIMIT 3;
--┌─────────┐
--│ id │
--│ varchar │
--├─────────┤
--│ fresh-a │
--│ fresh-b │
--└─────────┘
-- Without LIMIT the result is correct, so the Top-N optimizer is involved.
SELECT id FROM pg.example ORDER BY ts NULLS FIRST, id;
--┌───────────┐
--│ id │
--│ varchar │
--├───────────┤
--│ fresh-a │
--│ fresh-b │
--│ stamped-b │
--│ stamped-a │
--└───────────┘
-- Disabling Top-N also gives the correct 3 rows.
SET disabled_optimizers = 'top_n';
SELECT id FROM pg.example ORDER BY ts NULLS FIRST, id LIMIT 3;
--┌───────────┐
--│ id │
--│ varchar │
--├───────────┤
--│ fresh-a │
--│ fresh-b │
--│ stamped-b │
--└───────────┘
SET disabled_optimizers = '';
-- Show what is pushed to Postgres: the COPY ends in AND ("ts" IS NULL).
SET pg_debug_show_queries = true;
SELECT id FROM pg.example ORDER BY ts NULLS FIRST, id LIMIT 3;
--BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ
--
--COPY (SELECT "id", "ts" FROM "public"."example" WHERE ("ts" IS NULL)) TO STDOUT (FORMAT "binary");
--
--COMMIT
--
--┌─────────┐
--│ id │
--│ varchar │
--├─────────┤
--│ fresh-a │
--│ fresh-b │
--└─────────┘
SET pg_debug_show_queries = false;
-- Same data in a plain DuckDB table behaves correctly.
CREATE TABLE local_example AS SELECT * FROM pg.example;
SELECT id FROM local_example ORDER BY ts NULLS FIRST, id LIMIT 3;
--┌───────────┐
--│ id │
--│ varchar │
--├───────────┤
--│ fresh-a │
--│ fresh-b │
--│ stamped-b │
--└───────────┘
```
### OS:
MacOS
### PostgreSQL Version:
18
### DuckDB Version:
1.5.5
### DuckDB Client:
CLI
### Full Name:
Melker Veltman
### Affiliation:
Pickel AB
### Have you tried this on the latest `main` branch?
- [x] I agree
### Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?
- [x] I agree
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Postgres scanner path and the Top-N optimizer implicated by the reproduction. Run the Docker/PostgreSQL SQL example with pg_debug_show_queries enabled, then trace why ORDER BY ts NULLS FIRST LIMIT 3 pushes ts IS NULL. Done means the query returns fresh-a, fresh-b, and stamped-b while preserving correct behavior without LIMIT and with Top-N enabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, docker, postgresql, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100