SyntheticSource option forceNumInitialBundles is not optional in all cases
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
It fails when numRecords is between 1 and 3.
I noticed it by running SyntheticDataPublisher:
```
./gradlew :beam-sdks-java-load-tests:run -PloadTest.args='--pubSubTopic=projects/apache-beam-io-testing/topics/loadtests
--sourceOptions={"numRecords":2}' -PloadTest.mainClass="org.apache.beam.sdk.loadtests.SyntheticDataPublisher"
```
The exception:
```
Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at org.apache.beam.sdk.io.synthetic.BundleSplitter.getRelativeBundleSizes(BundleSplitter.java:60)
at org.apache.beam.sdk.io.synthetic.BundleSplitter.getBundleSizes(BundleSplitter.java:39)
at org.apache.beam.sdk.io.synthetic.SyntheticBoundedSource.split(SyntheticBoundedSource.java:146)
at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:214)
at org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:89)
at org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:76)
at org.apache.beam.runners.direct.ExecutorServiceParallelExecutor.start(ExecutorServiceParallelExecutor.java:155)
at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:208)
at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:67)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
at org.apache.beam.sdk.loadtests.SyntheticDataPublisher.main(SyntheticDataPublisher.java:150)
```
Imported from Jira [BEAM-7497](https://issues.apache.org/jira/browse/BEAM-7497). Original Jira may contain additional context.
Reported by: kasiak.
Contributor guide
Assessment
This issue has not been assessed yet.