apache / apache/doris

[Stream][Binlog] Override scan-state comparison for OlapStreamScan

Open
#65,878 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

### Description

While reviewing PR #65472, I found that `LogicalOlapTableStreamScan` has stream-specific state that is relevant to scan equivalence, especially `StreamReadMode` (`INCREMENTAL`, `SNAPSHOT`, and `RESET`). The class includes `readMode` in `equals()`, but it does not provide a stream-specific `hasSameScanState()` override.

After PR #65472 makes scan-state comparison relevant to plan rewrites such as `PullUpJoinFromUnionAll`, two stream scans may be compared through the generic OLAP scan-state logic without explicitly checking stream-specific state. If scans with different stream read modes are treated as equivalent, a rewrite could reuse or merge a scan with different stream semantics.

### Requested follow-up

Please add an override for `hasSameScanState()` in `LogicalOlapTableStreamScan` (or the corresponding `OlapStreamScan` abstraction) and include all stream-specific state that affects execution. Add FE unit tests covering:

- equal stream scans with the same read mode;
- different read modes;
- any selected-partition or stream-lowering state that must participate in scan equivalence.

This is a source-level follow-up to [PR #65472](https://github.com/apache/doris/pull/65472). The direct stream/binlog runtime impact still needs an end-to-end reproduction; this issue is opened to ensure the missing override and its contract are resolved explicitly.

**上面这个PR 会先pick 到4.1 分支,但是4.1分支现在不支持stream, 所以要等后面4.1支持了stream之后, 再单独pick stream scan的overrite 到4.1 分支上**

Related tracking issues:

- [#65265](https://github.com/apache/doris/issues/65265)
- [#65418](https://github.com/apache/doris/issues/65418)

Contributor guide

Open the contributing guide

Research direction

Start with LogicalOlapTableStreamScan and the generic hasSameScanState() implementation, then compare its stream-specific state with the existing equals() logic. Add FE unit tests for matching and differing StreamReadMode values, plus selected-partition or stream-lowering state identified during the review. Done means stream scans with different execution state are not considered equivalent.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.