Improve the logic to record intermediate results for multi-casting
Open
performance
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
See the discussion here: https://github.com/citusdata/citus/pull/3410#discussion_r369527510
Basically, we're using `list_append_unique_int()` for `#shardPlacements * # workerNodes` for each intermediate result in the query.
Especially with #3410, the optimization we had where we had short-circuits when we reach all nodes may likely to be hit less times, since we added `writeToLocalFile` check.
Some ideas we've had to use HTAB or bitmap, and run benchmarks.
Contributor guide
Assessment
This issue has not been assessed yet.