apache / apache/texera

Flaky ReconfigurationIntegrationSpec: small CSV source can finish before pause

Open
#5,913 5 comments 0 reactions 1 assignee Claimed by @aglinxinyuan View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Task Summary
`ReconfigurationIntegrationSpec`'s two CSV-sourced tests use a 100-row source (`smallCsvScanOpDesc`). The run can reach COMPLETED before the `pauseWorkflow` round-trip takes effect; when it does, `PauseHandler.pauseWorkflow` finds no running regions and emits the current state (COMPLETED) rather than PAUSED, so `TestUtils.shouldReconfigure`'s `Await.result(pausedReached, 10.seconds)` times out and the test fails. The existing `Retries` mixin does not help because the race recurs on retry. Fix by using `mediumCsvScanOpDesc` (100k rows) so the run stays in flight when pause lands, matching the non-flaky sibling specs `ReconfigurationSpec` and `PauseSpec`. Surfaced on the `amber-integration` CI job (e.g. run 28017649934).
### Task Type
- [x] Test

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.