apache / apache/beam

[Feature Request]: Document that file-based sinks return a PCollection of output filenames

Open
#25,769 0 comments 0 reactions 0 assignees View on GitHub
awaiting triage io new feature P2 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What would you like to happen?

It appears that the filebased sinks, such as `WriteToText` output a PCollection that is a list of files: https://github.com/apache/beam/blob/36486447e4d07af5076830ca1e331a6b61f14986/sdks/python/apache_beam/io/filebasedsink.py#L355

This is not clear from the documentation, for example, the docstring on WriteToText doesn't clarify the nature of output PCollection in https://beam.apache.org/releases/pydoc/2.45.0/apache_beam.io.textio.html#apache_beam.io.textio.WriteToText .

### Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

### Issue Components

- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [X] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Research direction

Start with sdks/python/apache_beam/io/filebasedsink.py around line 355 and the WriteToText docstring in apache_beam.io.textio. Clarify in the relevant documentation that file-based sinks return a PCollection containing output filenames, then check the generated Python API documentation to confirm the behavior is visible.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.