Two retry loops still hand-rolled: URLFetchUtil retries without delay, PythonProxyClient waits a constant delay
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Task Summary
Two retry loops were left hand-rolled when #7119 consolidated the rest onto `RetryUtil.withBackoff`, because converting them changes runtime behavior rather than just structure. Each needs a decision before it can move.
**1. `URLFetchUtil.getInputStreamFromURL`** (`common/workflow-operator`)
```scala
def getInputStreamFromURL(urlObj: URL, retries: Int = 5): Option[InputStream]
```
- Retries 5x with **no delay at all**, and returns `Option` rather than throwing.
- Adopting the util's backoff means a dead URL costs ~3 s (200+400+800+1600) before giving up, where today it fails immediately. That is arguably the fix — an unthrottled 5x retry against a remote host is a small retry storm — but it changes how long a URL-scan operator takes to fail.
- The `Option` return is a one-line adapter (`Try(...).toOption`), so the delay is the only real question.
**2. `PythonProxyClient`'s Flight-connect loop** (`amber`)
```scala
while (!connected && tryCount <= MAX_TRY_COUNT) { ... Thread.sleep(UNIT_WAIT_TIME_MS) ... }
```
- Waits a **constant** `UNIT_WAIT_TIME_MS` between attempts, not a doubling one, so the util needs a delay-multiplier knob (`1` = constant) before it can host this.
- Closes the Flight client between attempts; that per-attempt cleanup has to move into the operation body.
- Throws `WorkflowRuntimeException` on give-up, where the util wraps in `RuntimeException`. Either the util grows a give-up hook, or this call site accepts the different type — worth deciding deliberately, since worker-bring-up failures are surfaced to users.
Both are candidates, not obligations: if the answer for either is "the current behavior is what we want", closing this with that note recorded is a fine outcome.
### Task Type
- [x] Refactor / Cleanup
Contributor guide
Assessment
This issue has not been assessed yet.