Failing the job when the temporary file is accidentally deleted by another job in FileBasedSink
Open
bug
files
io
java
P3
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
There was an issue that a temporary directory name is not unique in FileBasedSink (https://issues.apache.org/jira/browse/BEAM-7689) and we've changed the naming algorithm from timestamp to UUID. However, it looks like UUID is still not enough to distinguish two separate temporary directories from different jobs. We may need to fail the job when the temporary file is accidentally deleted by another job.
Imported from Jira [BEAM-8289](https://issues.apache.org/jira/browse/BEAM-8289). Original Jira may contain additional context.
Reported by: heejong.
Contributor guide
Assessment
This issue has not been assessed yet.