kestra-io / kestra-io/plugin-debezium

core: `Deleted.NULL` is documented but never implemented in ChangeConsumer

Open
#201 0 comments 0 reactions 0 assignees View on GitHub
area/plugin good first issue
Dominant language
Java
Stars
5
Forks
11
Avg merge
2d 5h
Merged PRs (30d)
9

Description

## Summary

The `deleted` property on every Debezium task and trigger documents three modes — `ADD_FIELD`, `NULL`, and `DROP` — but `ChangeConsumer` only implements two. `Deleted.NULL` is neither filtered nor annotated anywhere in the consumer, so selecting it silently produces behaviour that does not match its documented description.

Surfaced during the documentation review in #199, which deliberately left code changes out of scope.

## Actual Behaviour

`plugin-debezium/src/main/java/io/kestra/plugin/debezium/AbstractDebeziumInterface.java` documents the enum as:

```
- `ADD_FIELD`: Add a deleted field as boolean.
- `NULL`: Send a row with all values as null.
- `DROP`: Don't send deleted row.
```

`ChangeConsumer` branches on the enum in exactly two places:

- `plugin-debezium/src/main/java/io/kestra/plugin/debezium/ChangeConsumer.java:201` — `... == Deleted.DROP`
- `plugin-debezium/src/main/java/io/kestra/plugin/debezium/ChangeConsumer.java:264` — `... == Deleted.ADD_FIELD`

There is no `NULL` branch. A user who sets `deleted: NULL` gets whatever the fall-through path produces, not a documented "row with all values as null" contract. The value is accepted by validation and produces no warning.

## Expected Behaviour

One of:

- **Implement it** — add the `NULL` branch to `ChangeConsumer` so a delete event emits a row whose values are all null, matching the documentation; or
- **Correct the contract** — if the fall-through behaviour is in fact what `NULL` was always meant to do, rewrite the `@Schema` description to state what actually happens; or
- **Remove the value** — if `NULL` was never meaningful, deprecate and drop it.

The first option matches the documented intent and is the least surprising for existing users; the third is a breaking change for any flow that already sets `deleted: NULL`.

## Reproducer

```yaml
id: debezium_deleted_null
namespace: company.team

tasks:
- id: capture
type: io.kestra.plugin.debezium.mysql.Capture
snapshotMode: NEVER
hostname: 127.0.0.1
port: "3306"
username: "{{ secret('MYSQL_USERNAME') }}"
password: "{{ secret('MYSQL_PASSWORD') }}"
serverId: "123456789"
deleted: NULL
maxRecords: 100
```

Delete a row in a captured table, then inspect the captured output: the emitted record does not match the documented "all values as null" shape.

## Logs / Stack Trace

No logs available — no error is raised; the behaviour is silently different from the documentation.

## Environment

- **Kestra version**: 1.3.16 (`kestraVersion` in `gradle.properties`)
- **Plugin version**: 1.4.7
- **Deployment**: n/a — source-level defect, reproducible on any deployment

## Acceptance Criteria

- [ ] A decision is recorded on which of the three resolutions applies
- [ ] `ChangeConsumer` and the `@Schema` description on `AbstractDebeziumInterface` agree on what `NULL` does
- [ ] A unit test covers the `deleted: NULL` path against a delete event and asserts the emitted row shape
- [ ] If the value is removed or its meaning changes, the breaking change is documented in the release notes

## Additional Context

Flagged in the review of #199 (documentation-only PR). The description lives in `AbstractDebeziumInterface`, a file that PR already edits, so the mismatch is visible on every database module's generated docs — the `deleted` property is inherited by all six connectors.

---
*[View as Artifact](https://claude.ai/code/artifact/a5050e50-116c-49ed-8d17-807a94c941c8)*

Contributor guide

No contributing guide indexed for this repository

Research direction

Read plugin-debezium/src/main/java/io/kestra/plugin/debezium/ChangeConsumer.java at the documented enum branches and compare it with the @Schema description in plugin-debezium/src/main/java/io/kestra/plugin/debezium/AbstractDebeziumInterface.java. Locate existing ChangeConsumer unit-test coverage, decide which NULL contract applies, and add coverage for a delete event; done when the implementation, schema, and test agree.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.