apache / apache/doris

Broadcast join wrongly chosen for un-analyzed tables, causing OOM

Open
#65,899 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

### Description

When a table (Olap or external: Hive/Iceberg/etc.) has not been analyzed, its row count is propagated as `UNKNOWN_ROW_COUNT` (-1). `StatsCalculator` clamps this to 1 row so the cost model can keep working, but `JoinUtils.checkBroadcastJoinStats` then sees "1 row" and picks the table as the broadcast join build side. For a large un-analyzed table this materializes the full table on every BE node and OOMs.

### Steps to reproduce

1. Create a large table and do **not** run `ANALYZE`.
2. Join it against another table with a session that allows broadcast joins.
3. Observe the planner picks a broadcast join; runtime OOMs.

### Expected behavior

The optimizer should refuse broadcast when the build-side row count was clamped from unknown, falling back to a shuffle join.

### Actual behavior

Broadcast join is chosen, leading to OOM.

Contributor guide

Open the contributing guide

Research direction

Start by tracing how StatsCalculator represents UNKNOWN_ROW_COUNT and how JoinUtils.checkBroadcastJoinStats consumes the clamped value. Reproduce with a large unanalyzed table joined under a session that allows broadcast joins, then verify the planner refuses broadcast and selects a shuffle join without causing runtime OOM.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.