testcontainers / testcontainers/testcontainers-java

RemoteDockerImage should fail immediately on fatal errors, ability to specify a custom pull retry timeout

Open
#3,829 1 comment 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
8.7k
Forks
1.9k
Avg merge
2d 17h
Merged PRs (30d)
9

Description

Right now RemoteDockerImage will try to re-pull container image on error until the default timeout of 2 minutes has been reached.

This is a reasonable thing to do to work around various I/O and network issues, but we should still allow user to specify a custom timeout and update the code to fail immediately on fatal errors (e.g. credential helper not available or invalid credentials configured).

Background, Context

If you utilize Docker image from some private registry (e.g. ECR, DockerHub, Quay, etc.) and credential helper for that registry is not available or credentials configured correctly, that could will try to retry up to 2 minutes before failing.

Exception similar to this one will be visible in the logs:

2021-02-23T12:34:35.050+0100 [DEBUG] [TestEventLogger]          at java.base/java.lang.Thread.run(Thread.java:834)
2021-02-23T12:34:35.051+0100 [DEBUG] [TestEventLogger]     [Test worker] WARN 🐳 [fooo.dkr.ecr.us-east-1.amazonaws.com/ingestion-pipeline:local-redis-cluster] - Retrying pull for image: foo.dkr.ecr.us-east-1.amazonaws.com/bar:foo (112s remaining)
2021-02-23T12:34:35.530+0100 [DEBUG] [TestEventLogger]     [docker-java-stream--1357434991] ERROR com.github.dockerjava.api.async.ResultCallbackTemplate - Error during callback
2021-02-23T12:34:35.530+0100 [DEBUG] [TestEventLogger]     com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"Get \"https://foo.dkr.ecr.us-east-1.amazonaws.com/v2/bar/foo\": no basic auth credentials"}

Proposed Change

I proposed updating that code to fail immediately if no basic auth credentials message is returned (https://github.com/Kami/testcontainers-java/blob/master/core/src/main/java/org/testcontainers/images/RemoteDockerImage.java#L87). I know that checking the error message is not the most robust thing, but it's probably the best we can do here.

Technically, Docker Java client could also be updated to throw some more specific exception in that case, but that would require more work.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with core/src/main/java/org/testcontainers/images/RemoteDockerImage.java, particularly the retry logic around the referenced line 87, and trace how Docker pull errors are surfaced. Done means callers can provide a custom pull retry timeout and fatal credential errors such as “no basic auth credentials” fail without waiting for the default timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java
Domain
devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.