citusdata / citusdata/citus

Improve the logic to record intermediate results for multi-casting

Open
#3,420 0 comments 0 reactions 0 assignees View on GitHub
performance
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

See the discussion here: https://github.com/citusdata/citus/pull/3410#discussion_r369527510

Basically, we're using `list_append_unique_int()` for `#shardPlacements * # workerNodes` for each intermediate result in the query.

Especially with #3410, the optimization we had where we had short-circuits when we reach all nodes may likely to be hit less times, since we added `writeToLocalFile` check.

Some ideas we've had to use HTAB or bitmap, and run benchmarks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.