ClickHouse / ClickHouse/ClickHouse

incorrect peak memory accounting during grace_hash

Open
#50,821 1 comment 0 reactions 0 assignees View on GitHub
comp-joins external usability
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

```sql
create table A (A Int64, B Int64, S String) Engine=MergeTree order by A
as select number,number, toString(arrayMap(i->cityHash64(i*number), range(100))) from numbers(1e7);

SET join_algorithm = 'grace_hash', grace_hash_join_initial_buckets=4, grace_hash_join_max_buckets=4, send_logs_level='debug';

select * from A a join A as b on a.A = b.A format Null;

Query id: f60f1d1a-06ec-47ed-bd4d-fd5e7a4c2233

Current memory usage (total): 7.00 GiB.
Current memory usage (for query): 7.00 GiB.
Current memory usage (total): 8.00 GiB.
Current memory usage (for query): 8.00 GiB.
...
Current memory usage (for query): 12.18 GiB.
Current memory usage (total): 12.20 GiB.
...
Current memory usage (total): 16.00 GiB.
Current memory usage (total): 17.37 GiB.
Current memory usage (total): 18.71 GiB.
Current memory usage (total): 19.00 GiB.
Current memory usage (total): 20.41 GiB.
Current memory usage (total): 21.35 GiB. -------------------------<<<<<<<<<<<---------
...
Peak memory usage (for query): 12.18 GiB. ?????

select formatReadableSize(memory_usage) from system.query_log where query_id = 'f60f1d1a-06ec-47ed-bd4d-fd5e7a4c2233' and type=2;
┌─formatReadableSize(memory_usage)─┐
│ 12.18 GiB │
└──────────────────────────────────┘
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.