Replace fixed Thread.sleep calls in SyncExecutionResource with a bounded result-readiness poll
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Task Summary
The synchronous execution endpoint (`SyncExecutionResource::executeWorkflowSync`) uses two unconditional `Thread.sleep(500)` calls to wait for the result writer to flush before reading results, adding ~1s of latency to every run. The engine already commits the result writer (via the worker's blocking `writerThread.join()`) before an operator's `COMPLETED` state becomes observable, so the data is on disk by the time we read it. Replace both sleeps with a bounded readiness-poll that compares each target operator's committed result-storage count against its expected output count from stats, with early-exit and a short timeout cap.
### Task Type
- [x] Performance
Contributor guide
Research direction
Start at SyncExecutionResource::executeWorkflowSync and trace how target operators expose committed result-storage counts and expected output counts from stats. Replace the two fixed waits with a bounded readiness poll that exits early when counts match and stops at a short timeout; verify that completed synchronous executions still return their results without the fixed latency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- api, backend, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100