apache / apache/seatunnel

[Bug] [Flink] Schema change events do not refresh all parallel sink writers

Open
#11,752 4 comments 0 reactions 1 assignee Claimed by @dybyte View on GitHub
bug flink
Dominant language
Java
Stars
9.7k
Forks
2.4k
Avg merge
3d 13h
Merged PRs (30d)
203

Description

### Search before asking

- [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.

### Description

## Description

When schema evolution is enabled and the sink parallelism is greater than one, a schema change event may reach only one downstream sink writer.

The writer that receives the event updates its local schema, while the other parallel writers continue using the previous schema. If subsequent records containing the evolved schema are routed to those stale writers, they may be written incorrectly or cause the job to fail.

## Reproduction

A reproducible pipeline has the following structure:

- MySQL CDC source with parallelism 1
- JDBC sink with parallelism greater than 1
- Add a column to the source table while the pipeline is running
- Insert or update records containing the new column

Only the sink writer receiving the schema change event refreshes its local schema. Other sink writers may continue processing records using the old schema.

## Expected behavior

A schema change should refresh the local schema state of every relevant parallel sink writer before records using the new schema are processed.

The implementation should also ensure that applying the external schema change remains safe when multiple sink writers are involved.

## Actual behavior

The schema change event is routed to only one downstream sink writer. Other sink writers remain stale and may later process records with a schema that they do not recognize.

## Impact

This is a correctness issue for schema evolution pipelines with parallel sinks. Depending on the connector and record distribution, it may cause:

- Missing or incorrectly mapped column values
- Write failures after schema evolution
- Non-deterministic behavior depending on which sink writer receives each record
- Tests passing accidentally when all evolved records are routed to the writer that received the schema change

## Proposed direction

- Ensure that schema changes are propagated to every relevant parallel sink writer.
- Ensure that all writers refresh their local schema before processing records with the evolved schema.
- Preserve correct ordering between pre-change records, the schema change, and post-change records.
- Avoid unsafe concurrent or duplicate external schema changes.
- Add an E2E regression test with multiple sink writers that verifies post-change records can be processed by writers other than the one that initially handles the schema change.

## Scope

This issue focuses on stale local schema state in parallel sink writers during schema evolution.

### Usage Scenario

_No response_

### Related issues

_No response_

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.