spring-projects / spring-projects/spring-data-couchbase

Bucket may not be opened when applications starts

Open
#1,070 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: enhancement
Dominant language
Java
Stars
284
Forks
197
PR merge metrics
No merged PRs in 30d

Description

I noticed this while working with the integration tests. @SpringBootTest + @Testcontainers. Some builds on CI were failing with timeout exception when trying to do repository.save() and CB error was

om.couchbase.client.core.error.AmbiguousTimeoutException: UpsertRequest, Reason: TIMEOUT
{"cancelled":true,"completed":true,"coreId":"0x3b57a1700000003","idempotent":false
,"reason":"TIMEOUT","requestId":161,"requestType":"UpsertRequest","retried":19
,"retryReasons":["BUCKET_NOT_AVAILABLE"], ...

I've added buckets printout right before save operation like this:

var couchbaseClientFactory = context.getApplicationContext().getBean(CouchbaseClientFactory.class);
couchbaseClientFactory.getCluster().buckets().getAllBuckets()
    .forEach((bucket, settings) -> log.info("Test Bucket: [{}]; Settings: [{}]; Healthy: [{}]", bucket, settings, settings.healthy()));

and it showed that the bucket is there and is healthy. But save() fails.

Following through the code from the client factory down to where the bucket opens, it looks like opening a bucket is an async operation that is not guaranteed to finish (or rather guaranteed not to finish) when the method exits:

SimpleCouchbaseClientFactory:
this.bucket = cluster.get().bucket(bucketName);

Cluster:
return bucketCache.computeIfAbsent(bucketName, n -> new Bucket(asyncCluster.bucket(n)));

AsyncCluster:
return bucketCache.computeIfAbsent(bucketName, n -> {
  core.openBucket(n);
  return new AsyncBucket(n, core, environment.get());
});

Core:
configurationProvider
  .openBucket(name)
  .subscribe(v -> {}, t -> {}, () -> eventBus.publish(new BucketOpenedEvent(Duration.ofNanos(System.nanoTime() - start), coreContext, name)));

.subscribe is async here so the client factory may very well get a bucket instance that has not been opened yet.

I tried couchbaseClientFactory.getBucket().waitUntilReady(...) but it does not seem to work here and I don't see any code in waitUntilReady() that would ensure that bucket is open.

A workaround I came up with is the following:

couchbaseClientFactory.getCluster().core().configurationProvider().openBucket(BUCKET_NAME).block(WAIT_TIME);

It's not very nice and I block on Mono but it works for tests where context starts in less than a second and I need to make sure bucket is available when test cases start executing.

It'd be good to ensure in SimpleCouchbaseClientFactory that the bucket is indeed opened when context starts.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in SimpleCouchbaseClientFactory and trace bucket creation through Cluster, AsyncCluster, and Core.configurationProvider().openBucket(). Verify how application context startup currently exposes the bucket before the asynchronous open completes, then ensure startup does not proceed until the bucket is available and confirm repository.save() no longer encounters BUCKET_NOT_AVAILABLE timeouts.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.