duckdb / duckdb/duckdb-java

Could not write temp file and throw Invalid argument during BIG table sort

Open
#156 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
127
Forks
80
Avg merge
13h 49m
Merged PRs (30d)
48

Description

As you can see below, i have a table with 56 million rows, and attemping to perform sorting to "hierarchy_path" column will cause it to fail with invalid argument. i observed 400 + tmp files are being created on and off during the execution and after 5 minutes it goes off with error, i consider it crashed or at least not completing the SQL to create the table.

duckdb "/db/com.db"
v1.2.1 8e52ec4395
Enter ".help" for usage hints.
D select count(*) from asset_tree;
┌─────────────────┐
│ count_star() │
│ int64 │
├─────────────────┤
│ 56452767 │
│ (56.45 million) │
└─────────────────┘
D

duckdb "/db/com.db"
v1.2.1 8e52ec4395
Enter ".help" for usage hints.
D create table eric as
select * from asset_tree order by hierarchy_path;
49% ▕█████████████████████████████▍ ▏ IO Error:
Could not write file "/db/com.db.tmp/duckdb_temp_block-4611686018427598582.block": Invalid argument
D .quit

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.