duckdb / duckdb/duckdb-postgres

ORDER BY col NULLS FIRST ... LIMIT returns only NULL rows through the Postgres scanner

Open
#564 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
372
Forks
105
Avg merge
10h 19m
Merged PRs (30d)
18

Description

### What happens?

When running a query against a postgres DB with `ORDER BY col NULLS FIRST LIMIT x`, it returns only the null rows.

### To Reproduce

Start a postgres container locally
`docker run -it --rm -p 5432:5432 -e POSTGRES_PASSWORD=password postgres:18`
Then, in a duckdb console the following queries reproduces the issue:
```sql
ATTACH 'host=localhost port=5432 dbname=postgres user=postgres password=password' AS pg (TYPE postgres);
CREATE TABLE pg.example (id text, ts timestamptz);
INSERT INTO pg.example VALUES
('stamped-b', '2026-09-01 00:00:00+00'),
('stamped-a', '2026-09-01 01:00:00+00'),
('fresh-b', NULL),
('fresh-a', NULL);

-- Expected: fresh-a, fresh-b, stamped-b. Actual: fresh-a, fresh-b.
SELECT id FROM pg.example ORDER BY ts NULLS FIRST, id LIMIT 3;
--┌─────────┐
--│ id │
--│ varchar │
--├─────────┤
--│ fresh-a │
--│ fresh-b │
--└─────────┘

-- Without LIMIT the result is correct, so the Top-N optimizer is involved.
SELECT id FROM pg.example ORDER BY ts NULLS FIRST, id;
--┌───────────┐
--│ id │
--│ varchar │
--├───────────┤
--│ fresh-a │
--│ fresh-b │
--│ stamped-b │
--│ stamped-a │
--└───────────┘

-- Disabling Top-N also gives the correct 3 rows.
SET disabled_optimizers = 'top_n';
SELECT id FROM pg.example ORDER BY ts NULLS FIRST, id LIMIT 3;
--┌───────────┐
--│ id │
--│ varchar │
--├───────────┤
--│ fresh-a │
--│ fresh-b │
--│ stamped-b │
--└───────────┘
SET disabled_optimizers = '';

-- Show what is pushed to Postgres: the COPY ends in AND ("ts" IS NULL).
SET pg_debug_show_queries = true;
SELECT id FROM pg.example ORDER BY ts NULLS FIRST, id LIMIT 3;
--BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ
--
--COPY (SELECT "id", "ts" FROM "public"."example" WHERE ("ts" IS NULL)) TO STDOUT (FORMAT "binary");
--
--COMMIT
--
--┌─────────┐
--│ id │
--│ varchar │
--├─────────┤
--│ fresh-a │
--│ fresh-b │
--└─────────┘
SET pg_debug_show_queries = false;

-- Same data in a plain DuckDB table behaves correctly.
CREATE TABLE local_example AS SELECT * FROM pg.example;
SELECT id FROM local_example ORDER BY ts NULLS FIRST, id LIMIT 3;
--┌───────────┐
--│ id │
--│ varchar │
--├───────────┤
--│ fresh-a │
--│ fresh-b │
--│ stamped-b │
--└───────────┘

```

### OS:

MacOS

### PostgreSQL Version:

18

### DuckDB Version:

1.5.5

### DuckDB Client:

CLI

### Full Name:

Melker Veltman

### Affiliation:

Pickel AB

### Have you tried this on the latest `main` branch?

- [x] I agree

### Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

- [x] I agree

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Postgres scanner path and the Top-N optimizer implicated by the reproduction. Run the Docker/PostgreSQL SQL example with pg_debug_show_queries enabled, then trace why ORDER BY ts NULLS FIRST LIMIT 3 pushes ts IS NULL. Done means the query returns fresh-a, fresh-b, and stamped-b while preserving correct behavior without LIMIT and with Top-N enabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker, postgresql, sql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.