TFRecordIOIT fails on Spark and Flink
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
When run on Spark or Flink remote cluster, TFRecordIOIT fails with the following stacktrace:
```
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
org.apache.beam.sdk.Pipeline$PipelineExecutionException:
java.io.FileNotFoundException: No files found for spec: PREFIX_1534520885787*.
at org.apache.beam.runners.spark.SparkPipelineResult.beamExceptionFrom(SparkPipelineResult.java:66)
at
org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:99)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:87)
at
org.apache.beam.runners.spark.TestSparkRunner.run(TestSparkRunner.java:116)
at org.apache.beam.runners.spark.TestSparkRunner.run(TestSparkRunner.java:61)
at
org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:350)
at
org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:331)
at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:128)
Caused
by:
java.io.FileNotFoundException: No files found for spec: PREFIX_1534520885787*.
```
Imported from Jira [BEAM-5163](https://issues.apache.org/jira/browse/BEAM-5163). Original Jira may contain additional context.
Reported by: ŁukaszG.
Contributor guide
Assessment
This issue has not been assessed yet.