matrixorigin / matrixorigin/matrixone
[Bug]: RESUME CDC TASK cannot clear a target-table error while task state is Running
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
# Summary
`RESUME CDC TASK ` cannot recover a table-level CDC error when `SHOW CDC TASK` reports the task state as `running`.
This conflicts with the CDC guide's documented recovery flow:
- TC-501 says that a missing target table records an error while the task remains running.
- TC-502 says that, after the target is fixed, `RESUME CDC TASK` clears the error and synchronization resumes.
After restoring the target table, the regular `RESUME CDC TASK` command is rejected because the task is still `Running`. The table error remains and replication cannot continue through the documented command. The restart variant does recover, which isolates the issue to the normal resume/state transition path.
## Version
- MatrixOne `main`: `820ee12144439c5b63c03b261f4330ec0364fdb3`
- Isolated two-CN MatrixOne deployment
- Three independent reproductions
## Minimal reproduction
1. Create matching source and target tables and create a table-level CDC task with `NoFull=true`.
2. Wait until `SHOW CDC TASK` reports a checkpoint for the source table.
3. Verify one post-checkpoint source insert reaches the target.
4. Drop the target table, then insert another source row.
5. Wait for `SHOW CDC TASK` to expose a non-empty `err_msg` for the missing target table. The task state is `running`, as documented by TC-501.
6. Recreate the same target table.
7. Run:
```sql
RESUME CDC TASK cdc_lifecycle;
```
## Actual result
The command is rejected:
```text
internal error: Task cdc_lifecycle status can not be change, now it is Running
```
The table error remains in `SHOW CDC TASK` and the CDC task does not retry the restored target through the regular resume command.
As a control, the following command eventually clears the error and delivers both the row written while the target was absent and a later row:
```sql
RESUME CDC TASK cdc_lifecycle 'restart';
```
## Expected result
After the target table has been restored, the documented command:
```sql
RESUME CDC TASK cdc_lifecycle;
```
should clear the table error and resume processing from the recorded watermark, even though the task's top-level state remains `running` during the per-table error.
Alternatively, if a restart-only recovery model is intentional, the command should not be documented as the TC-502 recovery path and the SQL error should direct users to the required restart form.
## Reproduction evidence
All three runs observed the same sequence:
1. Missing target table produced a visible table-level error.
2. Normal `RESUME CDC TASK` was rejected with the task-state error above.
3. The error remained visible after the rejected command.
4. The restart form cleared the error and delivered the pending and subsequent source rows.
The earlier pause/resume check in the same isolated deployment also confirmed that ordinary pause recovery does eventually deliver the paused-period change; this issue is specific to a table error while the task reports `running`.
## Related issue
This is related to, but distinct from, #25177. That issue fixed the table-detector path that turns a permanent error into a failed task. This reproduction uses an already-running pipeline and the documented TC-501 behavior where a missing target table keeps the task in `running`; the broken behavior is the incompatibility between that state and the documented TC-502 `RESUME CDC TASK` recovery command.
Contributor guide
Assessment
This issue has not been assessed yet.