debezium / debezium/dbz

State leaks from aborted incremental snapshot to the next

Open
#1,563 6 comments 0 reactions 0 assignees View on GitHub
component/core-library type/bug
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

## Bug report

**What Debezium connector do you use and what version?**

debezium-connector-mysql 3.4.0

---

**What is the connector configuration?**

Not relevant

---

**What is the captured database version and mode of deployment?**

Running local mysql in docker/podman; mysql:8.0.26

---

**What behavior do you expect?**

After I start an incremental snapshot of all tables (with datacollection `.*`), then abort that incremental snapshot - I expect that I can start another incremental snapshot on a single table.

---

**What behavior do you see?**

After "immediately" aborting the snapshot of "all tables" and starting a new incremental snapshot of a single table, parts of the state from the first snapshot leaks into the next snapshot execution.

What I've seen is that io.debezium.pipeline.source.snapshot.incremental.AbstractIncrementalSnapshotChangeEventSource#addDataCollectionNamesToSnapshot is called with a signalPayload where `signalPayload.offsetContext.incrementalSnapshotContext` contains e.g. `chunkEndPosition`, `lastEventKeySent`, `maximumKey` etc from the first table in the previous snapshot execution. As a result the new snapshot execution resumes the snapshot from the maximumKey from another snapshot of _another table_, which is obviously not correct.

Also I observed that `total_rows_scanned` in the TABLE_SCAN_COMPLETED notification includes rows from the previous aborted snapshot.

---

**Do you see the same behaviour using the latest released Debezium version?**

_Ideally, also verify with the latest Alpha/Beta/CR version._

Yes, I have seen it in 3.1.x and the same behaviour is in 3.4.0 which is the latest version.

---

**Do you have the connector logs, ideally from start till finish?**

_You might be asked later to provide [DEBUG/TRACE](https://debezium.io/documentation/reference/stable/operations/logging.html) level log._

No, but I can provide DEBUG/TRACE logs on request.

---

**How to reproduce the issue using our [tutorial](https://github.com/debezium/debezium-examples/tree/main/tutorial) deployment?**

Idk, I have built my own rest api to trigger snapshots, but it is just a thin wrapper around the JmxSignalChannel from debezium. I saw some mentioning of a feature to send signals from debezium-ui in recent versions, but I have never used debezium-ui so not sure if this issue can be reproduced from there.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.