GoogleCloudPlatform / GoogleCloudPlatform/spring-cloud-gcp
Spring Boot Cloud Run - issues with Pub/Sub subscription after deployment
- Dominant language
- Java
- Stars
- 551
- Forks
- 349
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 14
Description
Hi,
I am having issues with message acknowledgement when deploying a new revision of a Spring Boot application in Cloud Run. The issue occurs when messages are being pulled (refer to image "Creating subscriptions") after deployment of a new instance.
Following are some images displaying the logs from the application of the following case:
1. Publishing a message that will intentionally fail and triggering retries.
2. Deploying a new revision that stops the message from failing
3. New instance started and message getting acknowledged
The issue occurs in step 3 with the error messages `failed to send operations` and `Permanent error invalid ack id message, will not resend`. These error messages started after enabling `exactlyOnceDelivery`.
I enabled this setting to avoid handling and acknowledging the same message twice - this issue seems to be somewhat resolved as it does not happen every time in this case. However, it has been possible for me to replicate after enabling this setting (See last image of logs).
I want to make sure that I do not handle and acknowledge the same message more than once and also fix these errors occurring related to acknowledging.
Logs before deployment with failed message retrying as expected:
Logs after deployment:
Issues with duplicated message pull around deployment of new instances. Both for retrying failed messages and newly published message only when retried/published short after deployment:
Context:
Spring Boot application with Google Cloud Run (min 1 instance / max 5 instances).
Dependencies:
```
com.google.cloud
spring-cloud-gcp-dependencies
6.2.3
pom
import
```
```
com.google.cloud
spring-cloud-gcp-starter-pubsub
```
```
org.springframework.boot
spring-boot-starter-web
```
application.properties:
```
freddy.pubsub.enabled=true
spring.cloud.gcp.pubsub.publisher.enable-message-ordering=true
spring.cloud.gcp.pubsub.publisher.endpoint=europe-west1-pubsub.googleapis.com:443
spring.cloud.gcp.pubsub.subscriber.min-duration-per-ack-extension=300
```
Creating subscriptions:
*Note: This is executed via an endpoint that is called on-demand.
Pulling Pub/Sub messages:
*Note: This is executed once Spring Boot context event `ApplicationReadyEvent` is published.
Topic in Google Cloud:
Subscription in Google Cloud:
Contributor guide
Assessment
This issue has not been assessed yet.