setting numshards dynamically for TextIO.write()
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
I go through code for TextIO. It uses WriteFiles in TypedWrite. Now, WriteFiles has the functionality withNumShards(ValueProvider) and [withSharding](https://beam.apache.org/releases/javadoc/2.9.0/org/apache/beam/sdk/io/WriteFiles.html#withSharding-org.apache.beam.sdk.transforms.PTransform-)([PTransform](https://beam.apache.org/releases/javadoc/2.9.0/org/apache/beam/sdk/transforms/PTransform.html)<[PCollection](https://beam.apache.org/releases/javadoc/2.9.0/org/apache/beam/sdk/values/PCollection.html)<[UserT](https://beam.apache.org/releases/javadoc/2.9.0/org/apache/beam/sdk/io/WriteFiles.html)\>,[PCollectionView](https://beam.apache.org/releases/javadoc/2.9.0/org/apache/beam/sdk/values/PCollectionView.html)\> sharding) but TextIO or its inner class TypedWrite doesn't have that. It would really be a great to expose that functionality to user.
Are there some issues in implementation that this is not already available?
Imported from Jira [BEAM-6721](https://issues.apache.org/jira/browse/BEAM-6721). Original Jira may contain additional context.
Reported by: aditya.guru149.
Contributor guide
Assessment
This issue has not been assessed yet.