apache / apache/beam

Multiple Instances of Beam Table During Query Planning

Open
#19,562 0 comments 0 reactions 0 assignees View on GitHub
dsl improvement P3 sql
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

When Calcite is planning the query it may ask the table provider for the same table multiple times and in different alternative query plans there might be different instances of the same table.

Since the row count estimation is stored in the table instances, each time that the row count estimation is called for a new instance, the table creates a new estimate. The estimation may take some time; therefore, this can potentially downgrade the performance (of planning) and increase the planning time.

There are two potential ways to solve this problem:

1- Make sure that the table providers do not create multiple instances for the same table. 

or

2- Keep the row count estimations in a common data structure or a static Map and reuse it in multiple instances when needed.

Imported from Jira [BEAM-7655](https://issues.apache.org/jira/browse/BEAM-7655). Original Jira may contain additional context.
Reported by: riazela.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.