apache / apache/iceberg-go

ci: retry transient Docker pulls during integration setup

Open
#1,535 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
463
Forks
232
Avg merge
2d 12h
Merged PRs (30d)
118

Description

Two integration jobs failed during setup when Docker Hub timed out while pulling images:

- https://github.com/apache/iceberg-go/actions/runs/30167050157/job/89701755026
- https://github.com/apache/iceberg-go/actions/runs/30167050070/job/89701754817

Both failed before tests ran. Their Spark 4 counterparts passed.

Integration setup currently runs docker compose up -d --wait once, so a temporary registry timeout fails the whole job.

A few possible fixes:

1. Retry docker compose pull in CI with backoff, then start with --pull never. This seems safest and does not change local behavior.
2. Retry the full integration setup command. This is simpler, but may recreate a partially started environment.
3. Limit concurrent image pulls and add backoff to reduce registry pressure.
4. Use a registry mirror or preloaded image cache.

Would you prefer the CI-only pull retry, or should the retry behavior live in the Makefile so it can also be used locally?

Happy to send a PR with the preferred approach.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the CI integration setup that runs `docker compose up -d --wait` and review the two linked failed job logs. Choose and document the retry scope with the maintainers; done means transient Docker Hub pull failures no longer fail setup while local behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, docker-compose, github-actions
Domain
ci-cd, devops, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.