apache / apache/doris

[Bug] REPLACE TABLE may skip binlog on master FE when only table-level binlog is enabled

Open
#66,571 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
522

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.

### Version

- Source cluster initially ran Doris 3.0.8
- FE was upgraded to Doris 3.1.4
- The missing REPLACE_TABLE binlog was observed on Doris 3.1.4

### What's Wrong?

When database-level binlog is disabled but the original table has binlog.enable=true, a swap=false REPLACE TABLE operation may not generate a REPLACE_TABLE binlog on the master FE.

The issue occurs when the current master FE has a cold BinlogConfigCache, for example after FE restart or master failover. During swap=false replacement, the master unregisters the original table from Catalog before BinlogManager.addReplaceTable() evaluates whether to write binlog. The event carries origTblId; the cold cache cannot resolve that removed table ID and returns false. The master then skips addBinlog(), so table-level CCR cannot observe the new REPLACE event through getBinlog. It only observes the previous dummy binlog.

FE may log:
~~~text
fail to get table. db: , table id:
~~~

### What You Expected?

If the original table was binlog-enabled before REPLACE TABLE, the master FE should always emit the corresponding REPLACE_TABLE binlog. The decision must not depend on whether the old table remains in Catalog or whether BinlogConfigCache is warm.

### How to Reproduce?

1. Enable the global binlog feature.
2. Disable DB-level binlog:
~~~sql
ALTER DATABASE db_name SET PROPERTIES ("binlog.enable" = "false");
~~~
3. Enable binlog on the original table:
~~~sql
ALTER TABLE origin_table SET ("binlog.enable" = "true");
~~~
4. Restart or fail over FE so that the current master has a cold BinlogConfigCache.
5. Execute:
~~~sql
ALTER TABLE origin_table REPLACE WITH TABLE replacement_table PROPERTIES ("swap" = "false");
~~~
6. Query binlog using the original table ID. The new REPLACE_TABLE event is absent.

### Anything Else?

Related CCR tracking issue: #66265.

Upstream PR #48919 fixes the separate DB-level-enabled case by changing the single-table anyEnable assignment to OR logic. It is insufficient for db binlog=false plus table binlog=true, because the original table configuration is already unavailable after unregistering the table.

Suggested fix: capture the original table BinlogConfig before unregistering it, persist the snapshot in ReplaceTableOperationLog, and use it when emitting and replaying REPLACE_TABLE binlog.

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Research direction

Start with BinlogManager.addReplaceTable(), addBinlog(), and getBinlog(), then trace ReplaceTableOperationLog through the swap=false replacement flow. Verify how the original table configuration is lost after unregistering it and how replay uses the operation log. Done means a table-level-enabled replacement emits and replays REPLACE_TABLE binlog with a cold BinlogConfigCache, while the reproduction steps no longer miss the event.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.