DockerComposeContainer: random failures with SocatContainer or ContainerisedDockerCompose

Versions

• testcontainers 1.7.3
• jdk1.8.0_172
• docker 18.03.1-ce-mac65
• docker-compose 1.21.1

Problems

• SocatContainer randomly fails to start. Using alpine/socat:1.0.3 with 5 minutes timeout for DockerComposeContainer#ambassadorContainer and withLocalCompose(true):

Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:214)
	at org.testcontainers.containers.DockerComposeContainer.startAmbassadorContainers(DockerComposeContainer.java:221)
	at org.testcontainers.containers.DockerComposeContainer.starting(DockerComposeContainer.java:131)
	at utils.TeamCityInDocker$Companion$instance$2.invoke(TeamCityInDocker.kt:59)
	at utils.TeamCityInDocker$Companion$instance$2.invoke(TeamCityInDocker.kt:36)
	at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
	at utils.TeamCityInDocker$Companion.getInstance(TeamCityInDocker.kt)
	... 47 more
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:83)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:207)
	... 53 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:279)
	at org.testcontainers.containers.GenericContainer.lambda$start$0(GenericContainer.java:209)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:76)
	... 54 more
Caused by: org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:51)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilTrue(Unreliables.java:95)
	at org.testcontainers.containers.startupcheck.StartupCheckStrategy.waitUntilStartupSuccessful(StartupCheckStrategy.java:26)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:251)
	... 56 more
Caused by: java.lang.RuntimeException: Not ready yet
	at org.rnorth.ducttape.unreliables.Unreliables.lambda$retryUntilTrue$1(Unreliables.java:97)
	at org.rnorth.ducttape.unreliables.Unreliables.lambda$retryUntilSuccess$0(Unreliables.java:41)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more

• ContainerisedDockerCompose randomly (but very frequently) fails to start. Using docker/compose:1.21.2 and withLocalCompose(false):

INFO 🐳 [docker/compose:1.21.2] - Pulling docker image: docker/compose:1.21.2. Please be patient; this may take some time but only needs to be done once.
INFO 🐳 [docker/compose:1.21.2] - Creating container for image: docker/compose:1.21.2
INFO 🐳 [docker/compose:1.21.2] - Starting container with ID: 0d2413d84cd32005eadfb9168a9181aa1f994d87ae633697a43e0b4e1e677901
INFO 🐳 [docker/compose:1.21.2] - Container docker/compose:1.21.2 is starting: 0d2413d84cd32005eadfb9168a9181aa1f994d87ae633697a43e0b4e1e677901
ERROR c.g.dockerjava.core.async.ResultCallbackTemplate - Error during callback
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:214)
	at org.testcontainers.containers.ContainerisedDockerCompose.invoke(DockerComposeContainer.java:529)
	at org.testcontainers.containers.DockerComposeContainer.runWithCompose(DockerComposeContainer.java:189)
	at org.testcontainers.containers.DockerComposeContainer.pullImages(DockerComposeContainer.java:137)
	at org.testcontainers.containers.DockerComposeContainer.starting(DockerComposeContainer.java:127)
	at utils.TeamCityInDocker$Companion$instance$2.invoke(TeamCityInDocker.kt:59)
	at utils.TeamCityInDocker$Companion$instance$2.invoke(TeamCityInDocker.kt:36)
	at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
	at utils.TeamCityInDocker$Companion.getInstance(TeamCityInDocker.kt)
	... 22 more
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:83)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:207)
	... 30 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:279)
	at org.testcontainers.containers.GenericContainer.lambda$start$0(GenericContainer.java:209)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:76)
	... 31 more
Caused by: com.github.dockerjava.api.exception.NotFoundException: {"message":"No such container: 0d2413d84cd32005eadfb9168a9181aa1f994d87ae633697a43e0b4e1e677901"}

	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:103)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)
	at org.testcontainers.shaded.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
	at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
	at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
	at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1342)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:934)
	at org.testcontainers.shaded.io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:593)
	at org.testcontainers.shaded.io.netty.channel.kqueue.KQueueDomainSocketChannel$KQueueDomainUnsafe.readReady(KQueueDomainSocketChannel.java:131)
	at org.testcontainers.shaded.io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:367)
	at org.testcontainers.shaded.io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:198)
	at org.testcontainers.shaded.io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:270)
	at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at org.testcontainers.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:748)

Exception gets thrown either in DockerComposeContainer#pullImages or in DockerComposeContainer#createServices. Debugging showed me that after executing CreateContainerCmd compose container just doesn't get created sometimes, docker container ps --all shows nothing.

Bắt đầu bằng cách đọc DockerComposeContainer#pullImages, DockerComposeContainer#createServices và ContainerisedDockerCompose, sau đó so sánh các luồng đó với lỗi khởi động của SocatContainer. Tái hiện vấn đề tạo container xảy ra không liên tục bằng các phiên bản và cấu hình được nêu trong báo cáo. Được xem là hoàn thành khi cả hai chế độ compose đều khởi động ổn định mà không gặp các lỗi thiếu container hoặc timeout đã được báo cáo.