Question: Accessing docker compose container services when using DinD

What are you trying to do?

We're currently trying to use DockerCompose class objects to run integration tests inside our CI env (gitlab using DinD service).
As the docker composed set of services is being spin inside a DinD service container, they're not directly accessible from the job container,

Where are you trying to do it?

from testcontainers.compose import DockerCompose
compose = DockerCompose(
         "tests/integration/compose", compose_file_name="docker-compose.yml", pull=True
     )
     compose.start()
     print("PublishedPort: "+compose.get_container(service_name="schema-registry").get_publisher().URL)

This prints out an inaccessible URL (from the CI job container standpoint)
The same code works fine on local setup.

Runtime environment

(GitLab Job CI env settings)

uname -a
echo "------"
docker info
echo "------"
poetry run python --version
echo "------"
poetry show --tree

Linux 694dc901af1e 6.5.0-1022-aws #22~22.04.1-Ubuntu SMP Fri Jun 14 16:31:00 UTC 2024 x86_64 x86_64 x86_64 inux 694dc901af1e 6.5.0-1022-aws #22~22.04.1-Ubuntu SMP Fri Jun 14 16:31:00 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
------
Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.0.0+unknown
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1
    Path:     /usr/local/lib/docker/cli-plugins/docker-compose

Server:
ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
errors pretty printing info
------
Path /dp-gitlab-migration-test/mms_receiver/external_modules/application/aws_logging for aws_logging does not exist
Skipping virtualenv creation, as specified in config file.
Python 3.11.6
------
Path /dp-gitlab-migration-test/mms_receiver/external_modules/application/aws_logging for aws_logging does not exist
Skipping virtualenv creation, as specified in config file.
aws-lambda-powertools 2.35.1 Powertools for AWS Lambda (Python) is a developer toolkit to implement Serverless best practices and increase developer velocity.
└── typing-extensions >=4.6.2,<5.0.0
aws-logging 1.0.0 A package that contain a reusable module for structure logging
├── boto3 ^1.17.98
│   ├── botocore >=1.34.35,<1.35.0
│   │   ├── jmespath >=0.7.1,<2.0.0
│   │   ├── python-dateutil >=2.1,<3.0.0
│   │   │   └── six >=1.5
│   │   └── urllib3 >=1.25.4,<2.1
│   ├── jmespath >=0.7.1,<2.0.0 (circular dependency aborted here)
│   └── s3transfer >=0.10.0,<0.11.0
│       └── botocore >=1.33.2,<2.0a.0 (circular dependency aborted here)
└── structlog ^22.3.0
black 23.3.0 The uncompromising code formatter.
├── click >=8.0.0
│   └── colorama *
├── mypy-extensions >=0.4.3
├── packaging >=22.0
├── pathspec >=0.9.0
└── platformdirs >=2
boto3 1.34.35 The AWS SDK for Python
├── botocore >=1.34.35,<1.35.0
│   ├── jmespath >=0.7.1,<2.0.0
│   ├── python-dateutil >=2.1,<3.0.0
│   │   └── six >=1.5
│   └── urllib3 >=1.25.4,<2.1
├── jmespath >=0.7.1,<2.0.0
└── s3transfer >=0.10.0,<0.11.0
    └── botocore >=1.33.2,<2.0a.0
        ├── jmespath >=0.7.1,<2.0.0
        ├── python-dateutil >=2.1,<3.0.0
        │   └── six >=1.5
        └── urllib3 >=1.25.4,<2.1
confluent-kafka 2.3.0 Confluent's Python client for Apache Kafka
fastavro 1.9.4 Fast read/write of AVRO files
polyfactory 2.15.0 Mock data generation factories
├── faker *
│   └── python-dateutil >=2.4
│       └── six >=1.5
└── typing-extensions >=4.6.0
psycopg 3.1.18 PostgreSQL database adapter for Python
├── typing-extensions >=4.1
└── tzdata *
psycopg-binary 3.1.18 PostgreSQL database adapter for Python -- C optimisation distribution
psycopg2-binary 2.9.9 psycopg2 - Python-PostgreSQL Database Adapter
pydantic 2.6.4 Data validation using Python type hints
├── annotated-types >=0.4.0
├── email-validator >=2.0.0
│   ├── dnspython >=2.0.0
│   └── idna >=2.0.0
├── pydantic-core 2.16.3
│   └── typing-extensions >=4.6.0,<4.7.0 || >4.7.0
└── typing-extensions >=4.6.1
pytest 7.2.2 pytest: simple powerful testing with Python
├── attrs >=19.2.0
├── colorama *
├── iniconfig *
├── packaging *
└── pluggy >=0.12,<2.0
pytest-cov 4.0.0 Pytest plugin for measuring coverage.
├── coverage >=5.2.1
└── pytest >=4.6
    ├── attrs >=19.2.0
    ├── colorama *
    ├── iniconfig *
    ├── packaging *
    └── pluggy >=0.12,<2.0
requests 2.31.0 Python HTTP for Humans.
├── certifi >=2017.4.17
├── charset-normalizer >=2,<4
├── idna >=2.5,<4
└── urllib3 >=1.21.1,<3
sqlalchemy 2.0.28 Database Abstraction Library
├── greenlet !=0.4.17
└── typing-extensions >=4.6.0
testcontainers 4.5.1 Python library for throwaway instances of anything that can run in a Docker container
├── docker *
│   ├── pywin32 >=304
│   ├── requests >=2.26.0
│   │   ├── certifi >=2017.4.17
│   │   ├── charset-normalizer >=2,<4
│   │   ├── idna >=2.5,<4
│   │   └── urllib3 >=1.21.1,<3
│   └── urllib3 >=1.26.0 (circular dependency aborted here)
├── typing-extensions *
├── urllib3 *
└── wrapt *
tox 4.4.8 tox is a generic virtualenv management and test command line tool
├── cachetools >=5.3
├── chardet >=5.1
├── colorama >=0.4.6
├── filelock >=3.10
├── packaging >=23
├── platformdirs >=3.1.1
├── pluggy >=1
├── pyproject-api >=1.5.1
│   └── packaging >=23.1
└── virtualenv >=20.21
    ├── distlib >=0.3.7,<1
    ├── filelock >=3.12.2,<4
    └── platformdirs >=3.9.1,<5

从 tests/integration/compose 中展示的 DockerCompose 流程开始，重点关注 compose.start()、get_container() 和 get_publisher()。在 GitLab DinD 环境中复现该示例，并将发布的 URL 与本地行为进行比较。完成的标准是确定是否可以使该 URL 从作业容器访问，或记录所需的受支持配置。