weaviate / weaviate/java-client
Enable parallel processing in synchronous pagination
Open
@bevzzz is already working on this.
Since Jul 4, 2025.
v6
- Dominant language
- Java
- Stars
- 34
- Forks
- 30
- PR merge metrics
- No merged PRs in 30d
Description
Java's Stream API features a built-in support for parallel stream processing: calling myList.stream().parallel() will distribute the workload across the available threads.
CursorSpliterator which underpins our pagination implementation is returning null from its trySplit() method, which the Stream model understands as "this cannot be processed in parallel".
We want to unlock this possibility for our users.
There are several possible strategies for splitting the stream of Weaviate objects. See this discussion for more details.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.