Question: Accessing docker compose container services when using DinD

What are you trying to do?

We're currently trying to use DockerCompose class objects to run integration tests inside our CI env (gitlab using DinD service).
As the docker composed set of services is being spin inside a DinD service container, they're not directly accessible from the job container,

Where are you trying to do it?

from testcontainers.compose import DockerCompose
compose = DockerCompose(
         "tests/integration/compose", compose_file_name="docker-compose.yml", pull=True
     )
     compose.start()
     print("PublishedPort: "+compose.get_container(service_name="schema-registry").get_publisher().URL)

This prints out an inaccessible URL (from the CI job container standpoint)
The same code works fine on local setup.

Runtime environment

(GitLab Job CI env settings)

uname -a
echo "------"
docker info
echo "------"
poetry run python --version
echo "------"
poetry show --tree

Linux 694dc901af1e 6.5.0-1022-aws #22~22.04.1-Ubuntu SMP Fri Jun 14 16:31:00 UTC 2024 x86_64 x86_64 x86_64 inux 694dc901af1e 6.5.0-1022-aws #22~22.04.1-Ubuntu SMP Fri Jun 14 16:31:00 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
------
Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.0.0+unknown
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1
    Path:     /usr/local/lib/docker/cli-plugins/docker-compose

Server:
ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
errors pretty printing info
------
Path /dp-gitlab-migration-test/mms_receiver/external_modules/application/aws_logging for aws_logging does not exist
Skipping virtualenv creation, as specified in config file.
Python 3.11.6
------
Path /dp-gitlab-migration-test/mms_receiver/external_modules/application/aws_logging for aws_logging does not exist
Skipping virtualenv creation, as specified in config file.
aws-lambda-powertools 2.35.1 Powertools for AWS Lambda (Python) is a developer toolkit to implement Serverless best practices and increase developer velocity.
└── typing-extensions >=4.6.2,<5.0.0
aws-logging 1.0.0 A package that contain a reusable module for structure logging
├── boto3 ^1.17.98
│   ├── botocore >=1.34.35,<1.35.0
│   │   ├── jmespath >=0.7.1,<2.0.0
│   │   ├── python-dateutil >=2.1,<3.0.0
│   │   │   └── six >=1.5
│   │   └── urllib3 >=1.25.4,<2.1
│   ├── jmespath >=0.7.1,<2.0.0 (circular dependency aborted here)
│   └── s3transfer >=0.10.0,<0.11.0
│       └── botocore >=1.33.2,<2.0a.0 (circular dependency aborted here)
└── structlog ^22.3.0
black 23.3.0 The uncompromising code formatter.
├── click >=8.0.0
│   └── colorama *
├── mypy-extensions >=0.4.3
├── packaging >=22.0
├── pathspec >=0.9.0
└── platformdirs >=2
boto3 1.34.35 The AWS SDK for Python
├── botocore >=1.34.35,<1.35.0
│   ├── jmespath >=0.7.1,<2.0.0
│   ├── python-dateutil >=2.1,<3.0.0
│   │   └── six >=1.5
│   └── urllib3 >=1.25.4,<2.1
├── jmespath >=0.7.1,<2.0.0
└── s3transfer >=0.10.0,<0.11.0
    └── botocore >=1.33.2,<2.0a.0
        ├── jmespath >=0.7.1,<2.0.0
        ├── python-dateutil >=2.1,<3.0.0
        │   └── six >=1.5
        └── urllib3 >=1.25.4,<2.1
confluent-kafka 2.3.0 Confluent's Python client for Apache Kafka
fastavro 1.9.4 Fast read/write of AVRO files
polyfactory 2.15.0 Mock data generation factories
├── faker *
│   └── python-dateutil >=2.4
│       └── six >=1.5
└── typing-extensions >=4.6.0
psycopg 3.1.18 PostgreSQL database adapter for Python
├── typing-extensions >=4.1
└── tzdata *
psycopg-binary 3.1.18 PostgreSQL database adapter for Python -- C optimisation distribution
psycopg2-binary 2.9.9 psycopg2 - Python-PostgreSQL Database Adapter
pydantic 2.6.4 Data validation using Python type hints
├── annotated-types >=0.4.0
├── email-validator >=2.0.0
│   ├── dnspython >=2.0.0
│   └── idna >=2.0.0
├── pydantic-core 2.16.3
│   └── typing-extensions >=4.6.0,<4.7.0 || >4.7.0
└── typing-extensions >=4.6.1
pytest 7.2.2 pytest: simple powerful testing with Python
├── attrs >=19.2.0
├── colorama *
├── iniconfig *
├── packaging *
└── pluggy >=0.12,<2.0
pytest-cov 4.0.0 Pytest plugin for measuring coverage.
├── coverage >=5.2.1
└── pytest >=4.6
    ├── attrs >=19.2.0
    ├── colorama *
    ├── iniconfig *
    ├── packaging *
    └── pluggy >=0.12,<2.0
requests 2.31.0 Python HTTP for Humans.
├── certifi >=2017.4.17
├── charset-normalizer >=2,<4
├── idna >=2.5,<4
└── urllib3 >=1.21.1,<3
sqlalchemy 2.0.28 Database Abstraction Library
├── greenlet !=0.4.17
└── typing-extensions >=4.6.0
testcontainers 4.5.1 Python library for throwaway instances of anything that can run in a Docker container
├── docker *
│   ├── pywin32 >=304
│   ├── requests >=2.26.0
│   │   ├── certifi >=2017.4.17
│   │   ├── charset-normalizer >=2,<4
│   │   ├── idna >=2.5,<4
│   │   └── urllib3 >=1.21.1,<3
│   └── urllib3 >=1.26.0 (circular dependency aborted here)
├── typing-extensions *
├── urllib3 *
└── wrapt *
tox 4.4.8 tox is a generic virtualenv management and test command line tool
├── cachetools >=5.3
├── chardet >=5.1
├── colorama >=0.4.6
├── filelock >=3.10
├── packaging >=23
├── platformdirs >=3.1.1
├── pluggy >=1
├── pyproject-api >=1.5.1
│   └── packaging >=23.1
└── virtualenv >=20.21
    ├── distlib >=0.3.7,<1
    ├── filelock >=3.12.2,<4
    └── platformdirs >=3.9.1,<5

Bắt đầu với luồng DockerCompose được示示 trong tests/integration/compose, đặc biệt là compose.start(), get_container() và get_publisher(). Tái hiện ví dụ trong môi trường GitLab DinD và so sánh URL được công bố với hành vi cục bộ. Hoàn thành có nghĩa là xác định được liệu URL có thể được truy cập từ job container hay không, hoặc ghi lại cấu hình được hỗ trợ cần thiết.