GoogleCloudPlatform / GoogleCloudPlatform/spring-cloud-gcp
Support Batch Message Producing Consumption in Spring Cloud Stream GCP Pub/Sub Binder
- Dominant language
- Java
- Stars
- 551
- Forks
- 349
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 14
Description
The Google Pub/Sub binder in Spring Cloud Stream currently lacks native support for **batch message producing and consuming**, features that are already supported in the Kafka binder and formally defined in the Spring Cloud Stream programming model. As Pub/Sub workloads scale, the absence of true batching leads to higher per-message overhead, reduced throughput, and increased operational cost for both producers and consumers.
For **consumers**, applications are required to process messages one by one or implement custom buffering logic. **While `maxFetchSize` can reduce RPC calls by fetching multiple records at once, it still does not allow actual batch processing, and unlike the Kafka binder—where batching is officially supported—attempting batch handling in Pub/Sub results in casting exceptions, confirming that the Pub/Sub binder does not support true batch consumption or parallel batch processing.**
For **producers**, messages must be published individually through the binder, even though Pub/Sub’s underlying client library supports batching at the publisher level.
The Spring Cloud Stream **core programming model** already defines batching constructs:
* **Batch Producers**
https://docs.spring.io/spring-cloud-stream/reference/spring-cloud-stream/producing-and-consuming-messages.html#batch-producers
* **Batch Consumers**
https://docs.spring.io/spring-cloud-stream/reference/spring-cloud-stream/producing-and-consuming-messages.html#batch-consumers
The Kafka binder implements these concepts with clear batch-consuming capabilities: https://docs.spring.io/spring-cloud-stream/reference/kafka/kafka-binder/consume-batches.html
### **Benefits**
* Consistent developer experience across binders.
* Significantly improved throughput for Pub/Sub workloads.
* Reduced application-side boilerplate and complexity.
* Full alignment with the Spring Cloud Stream programming model for batch producers and consumers.
Adding batch producing and consuming support to the Pub/Sub binder would greatly enhance performance and bring feature parity with the Kafka binder and core Spring Cloud Stream batching abstractions.
Contributor guide
Assessment
This issue has not been assessed yet.