Provide a better error message for 410 Gone errors from GCS
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
For example 410 errors might get raised here.
com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:431)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:289)
at com.google.cloud.dataflow.sdk.io.FileBasedSink$FileBasedWriter.close(FileBasedSink.java:571)
at com.google.cloud.dataflow.sdk.io.FileBasedSink$FileBasedWriter.close(FileBasedSink.java:474)
at com.google.cloud.dataflow.sdk.io.Write$Bound$WriteBundles.finishBundle(Write.java:202)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
Example client side error message that gives a better description to the end-user: "GCS upload of processed data transiently failed and will need to be retried. Some produced data might need to be recomputed"
Imported from Jira [BEAM-7549](https://issues.apache.org/jira/browse/BEAM-7549). Original Jira may contain additional context.
Reported by: chamikara.
Contributor guide
Assessment
This issue has not been assessed yet.