Retry on path validation errors
Open
dataflow
improvement
P3
runners
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 204
Description
DataflowRunner may fail to validate path prefix due to transient errors. Consider retrying on those.
`verifyPathIsAccessible` may include simple retry logic for GCS paths.
Imported from Jira [BEAM-3149](https://issues.apache.org/jira/browse/BEAM-3149). Original Jira may contain additional context.
Reported by: altay.
Contributor guide
Research direction
Start by locating DataflowRunner and its verifyPathIsAccessible method, then inspect how GCS path prefixes are currently validated and how transient failures are represented. Define the retry behavior and add coverage showing that a transient validation error is retried while a persistent error still fails.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, java
- Domain
- cloud, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100