matrixorigin / matrixorigin/matrixone
[Bug]: prepared AP runtime cache UT cleanup times out under WAL pressure
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
### Is there an existing issue for the same bug?
- [x] I checked the existing issues; no match was found.
### Branch Name
main
### Commit ID
2d79f8f833d170b899ce4d5748032545f65982ac (PR #28770 CI run)
### Other Environment Information
- CI run: https://github.com/matrixorigin/matrixone/actions/runs/34695875816
- Job: Matrixone CI execution / UT Test on Ubuntu/x86
- Failure occurred in `TestPreparedAPRuntimeCacheWorkspaceTransition/remote-only=true`.
### Actual Behavior
The UT failed during deferred cleanup at `pkg/tests/dml/prepared_cache_test.go:72` while calling `DebugUpdateCNWorkState` to restore the CN to `Working`. It returned `context deadline exceeded` / `internal error: debugUpdateCNWorkState`. Nearby logs show WAL append latency around 4.18 seconds, slow transaction commits, and missed logservice heartbeats. The sibling `remote-only=false` subtest passed.
### Expected Behavior
The test should restore its CN work state reliably under runner load, while keeping the draining-placement assertion and cleanup checks intact.
### Steps to Reproduce
1. Run the Linux/x86 UT workflow under high load.
2. Execute `TestPreparedAPRuntimeCacheWorkspaceTransition/remote-only=true`.
3. Observe whether the deferred restore completes after the test has put one CN into `Draining`.
### Additional information
This is one observed failure, not proof of a repeatable flake or a product defect. The failure is in cluster-state cleanup and coincides with WAL/heartbeat timeouts; PR #28770 does not modify prepared-plan caching or this test. Please investigate cleanup behavior and runner-pressure sensitivity without removing the placement or restoration assertions.
Contributor guide
Assessment
This issue has not been assessed yet.