apache / apache/datafusion

Add `return_empty_batch` flag to `LimitedBatchCoalescer`

Open
#18,859 4 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

After adding `BatchCoalescer`/`LimitedBatchCoalescer` to Nested Loop Join and Hash Join there is the same issue of having to create a flag to return an empty batch. If the coalescer does not produce any rows then an empty schema will be returned instead of an empty record batch with the join schema.

### Describe the solution you'd like

Add a flag to `LimitedBatchCoalescer` so that when `finish()` is called, it will check:

```
if rows_returned == 0 && coalescer.return_empty_batch {
return RecordBatch::empty()
}
```

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by locating LimitedBatchCoalescer and its finish() implementation, then inspect its use in the Nested Loop Join and Hash Join paths. Confirm how rows_returned and the join schema are handled; done means the new flag causes finish() to return an empty record batch with the join schema when no rows are produced.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.