ci: retry transient Docker pulls during integration setup
- Dominant language
- Go
- Stars
- 463
- Forks
- 232
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 118
Description
Two integration jobs failed during setup when Docker Hub timed out while pulling images:
- https://github.com/apache/iceberg-go/actions/runs/30167050157/job/89701755026
- https://github.com/apache/iceberg-go/actions/runs/30167050070/job/89701754817
Both failed before tests ran. Their Spark 4 counterparts passed.
Integration setup currently runs docker compose up -d --wait once, so a temporary registry timeout fails the whole job.
A few possible fixes:
1. Retry docker compose pull in CI with backoff, then start with --pull never. This seems safest and does not change local behavior.
2. Retry the full integration setup command. This is simpler, but may recreate a partially started environment.
3. Limit concurrent image pulls and add backoff to reduce registry pressure.
4. Use a registry mirror or preloaded image cache.
Would you prefer the CI-only pull retry, or should the retry behavior live in the Makefile so it can also be used locally?
Happy to send a PR with the preferred approach.
Contributor guide
Research direction
Start by inspecting the CI integration setup that runs `docker compose up -d --wait` and review the two linked failed job logs. Choose and document the retry scope with the maintainers; done means transient Docker Hub pull failures no longer fail setup while local behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, docker-compose, github-actions
- Domain
- ci-cd, devops, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100