apache / apache/beam

Add .withPipelineTempDirectory() Builder setting for file-based sinks

Open
#21,172 0 comments 0 reactions 0 assignees View on GitHub
files improvement io java P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Per the discussion in dev@ mailing list: [https://lists.apache.org/thread.html/r927907b17a33f672ee04826ba8d3cb7004d4c7e7c18608b3562f4e9c%40%3Cdev.beam.apache.org%3E](https://lists.apache.org/thread.html/r927907b17a33f672ee04826ba8d3cb7004d4c7e7c18608b3562f4e9c%40%3Cdev.beam.apache.org%3E)

 

adds a Builder setting for core file-based sinks to use the user-specified PipelineOptions#tempLocation as the temp directory for file writes (before the files are renamed to the final output destination).

Imported from Jira [BEAM-12948](https://issues.apache.org/jira/browse/BEAM-12948). Original Jira may contain additional context.
Reported by: clairemcginty.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.