testcontainers / testcontainers/testcontainers-java

[Bug]: Cannot connect to CosmosDB Container if Direct mode is used

Open
#5,518 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

modules/azure type/bug
Dominant language
Java
Stars
8.7k
Forks
1.9k
Avg merge
2d 17h
Merged PRs (30d)
9

Description

Module

Azure

Testcontainers version

1.17.2

Using the latest Testcontainers version?

Yes

Docker version
❯ docker version
Client:
 Cloud integration: v1.0.20
 Version:           20.10.10
 API version:       1.41
 Go version:        go1.16.9
 Git commit:        b485636
 Built:             Mon Oct 25 07:43:15 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.10
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.9
  Git commit:       e2f740d
  Built:            Mon Oct 25 07:41:30 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.11
  GitCommit:        5b46e404f6b9f661a205e28d59c982d3634148f8
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
What happened?

Hello,

I want to run the testcontainer using direct connection instead of gateway connection for Cosmos DB.
https://www.testcontainers.org/modules/azure/

After following this tutorial I was able to run on gateway mode, but when I try to run on direct I get Fail to reach global gateway [https://localhost:54950]

I've exposed the missing ports 10251, 10252, 10253, 10254 but the error still goes on.

When I use the image directly on Docker I am able to run the tests without any issues.
https://docs.microsoft.com/en-us/azure/cosmos-db/linux-emulator?tabs=sql-api%2Cssl-netstd21

Relevant log output
25-06-2022 14:16:17.549 [main] INFO  ?.m.com/cosmosdb/linux/azure-cosmos-emulator:latest].tryStart - Container mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest is starting: 35b1dfb27ae5fbd9011249ffd8cd7bb602d470387fb4d7142504e4d09f8f245c
25-06-2022 14:16:36.009 [main] INFO  ?.m.com/cosmosdb/linux/azure-cosmos-emulator:latest].tryStart - Container mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest started in PT18.625343S
25-06-2022 14:16:36.012 [main] INFO  c.a.c.i.ImplementationBridgeHelpers.setDirectConnectionConfigAccessor - Setting DirectConnectionConfigAccessor...
25-06-2022 14:17:25.608 [main] INFO  c.a.c.i.ImplementationBridgeHelpers.setCosmosClientBuilderAccessor - Setting CosmosClientBuilderAccessor...
25-06-2022 14:17:26.557 [main] INFO  c.a.c.i.RxDocumentClientImpl.<init> - Initializing DocumentClient [1] with serviceEndpoint [https://localhost:54950], connectionPolicy [ConnectionPolicy{httpNetworkRequestTimeout=PT1M, tcpNetworkRequestTimeout=PT5S, connectionMode=DIRECT, maxConnectionPoolSize=1000, idleHttpConnectionTimeout=PT1M, idleTcpConnectionTimeout=PT0S, userAgentSuffix='', throttlingRetryOptions=RetryOptions{maxRetryAttemptsOnThrottledRequests=9, maxRetryWaitTime=PT30S}, endpointDiscoveryEnabled=false, preferredRegions=null, multipleWriteRegionsEnabled=true, proxyType=null, inetSocketProxyAddress=null, readRequestsFallbackEnabled=true, connectTimeout=PT5S, idleTcpEndpointTimeout=PT1H, maxConnectionsPerEndpoint=130, maxRequestsPerConnection=30, tcpConnectionEndpointRediscoveryEnabled=true}], consistencyLevel [null], directModeProtocol [Tcp]
25-06-2022 14:17:26.956 [main] DEBUG c.a.c.i.GlobalEndpointManager.startRefreshLocationTimerAsync - registering a refresh in [300000] ms
25-06-2022 14:17:26.982 [cosmos-parallel-1] DEBUG c.a.c.i.GlobalEndpointManager.lambda$startRefreshLocationTimerAsync$11 - startRefreshLocationTimerAsync() - Invoking refresh, I was registered on [2022-06-25T14:17:26.956854]
25-06-2022 14:17:26.983 [cosmos-parallel-1] INFO  c.a.c.i.RxDocumentClientImpl.getDatabaseAccountFromEndpoint - Getting database account endpoint from https://localhost:54950
25-06-2022 14:17:27.061 [cosmos-parallel-1] INFO  c.a.c.i.ImplementationBridgeHelpers.setCosmosDiagnosticsAccessor - Setting CosmosDiagnosticsAccessor...
25-06-2022 14:18:27.380 [reactor-http-kqueue-1] WARN  r.n.http.client.HttpClientConnect.warn - [e00b6593-1, L:/127.0.0.1:55053 - R:localhost/127.0.0.1:54950] The connection observed an error
io.netty.handler.timeout.ReadTimeoutException: null
25-06-2022 14:18:27.383 [reactor-http-kqueue-1] ERROR c.a.c.i.RxGatewayStoreModel.lambda$toDocumentServiceResponse$3 - Network failure
io.netty.handler.timeout.ReadTimeoutException: null
25-06-2022 14:18:27.385 [reactor-http-kqueue-1] INFO  c.a.c.i.ImplementationBridgeHelpers.setCosmosExceptionAccessor - Setting CosmosExceptionAccessor...
25-06-2022 14:18:27.398 [reactor-http-kqueue-1] WARN  c.a.c.i.RxDocumentClientImpl.lambda$getDatabaseAccountFromEndpoint$147 - Failed to retrieve database account information. io.netty.handler.timeout.ReadTimeoutException
25-06-2022 14:18:27.459 [reactor-http-kqueue-1] ERROR c.a.c.i.GlobalEndpointManager.lambda$getDatabaseAccountFromAnyLocationsAsync$2 - Fail to reach global gateway [https://localhost:54950], [{"innerErrorMessage":null,"cosmosDiagnostics":{"userAgent":"azsdk-java-cosmos/4.31.0 MacOSX/12.4 JRE/11.0.12","activityId":null,"requestLatencyInMs":60332,"requestStartTimeUTC":"2022-06-25T12:17:27.064109Z","requestEndTimeUTC":"2022-06-25T12:18:27.396643Z","responseStatisticsList":[],"supplementalResponseStatisticsList":[],"addressResolutionStatistics":{},"regionsContacted":[],"retryContext":{"statusAndSubStatusCodes":null,"retryCount":0,"retryLatency":0},"metadataDiagnosticsContext":{"metadataDiagnosticList":null},"serializationDiagnosticsContext":{"serializationDiagnosticsList":null},"gatewayStatistics":{"sessionToken":null,"operationType":"Read","resourceType":"DatabaseAccount","statusCode":408,"subStatusCode":10002,"requestCharge":0.0,"requestTimeline":[{"eventName":"connectionCreated","startTimeUTC":"2022-06-25T12:17:27.068561Z","durationInMicroSec":271607},{"eventName":"connectionConfigured","startTimeUTC":"2022-06-25T12:17:27.340168Z","durationInMicroSec":20503},{"eventName":"requestSent","startTimeUTC":"2022-06-25T12:17:27.360671Z","durationInMicroSec":9624},{"eventName":"transitTime","startTimeUTC":"2022-06-25T12:17:27.370295Z","durationInMicroSec":60016179},{"eventName":"received","startTimeUTC":null,"durationInMicroSec":0}],"partitionKeyRangeId":null,"exceptionMessage":null,"exceptionResponseHeaders":"{x-ms-substatus=10002}"},"systemInformation":{"usedMemory":"95911 KB","availableMemory":"4098393 KB","systemCpuLoad":"(2022-06-25T12:18:01.957304Z 12,2%), (2022-06-25T12:18:06.957474Z 11,8%), (2022-06-25T12:18:11.953689Z 15,8%), (2022-06-25T12:18:16.956634Z 13,1%), (2022-06-25T12:18:21.956681Z 13,0%), (2022-06-25T12:18:26.956230Z 13,6%)","availableProcessors":12},"clientCfgs":{"id":1,"machineId":"uuid:dea84019-34f2-4369-8b57-dcb481beab0a","connectionMode":"DIRECT","numberOfClients":1,"connCfg":{"rntbd":null,"gw":"(cps:1000, nrto:PT1M, icto:PT1M, p:false)","other":"(ed: false, cs: false)"},"consistencyCfg":"(consistency: null, mm: true, prgns: [])"}}}]
Additional Information

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No repository file or test is named in the report. Start by locating the Azure module's Cosmos DB container implementation and comparing its direct-mode configuration with the linked emulator documentation and the listed ports; done means the reported direct-connection scenario works without the global-gateway timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java
Domain
cloud, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.