testcontainers / testcontainers/testcontainers-python

Can't connect to PostgreSQL container from another container when running on Mac as host

未关闭
#138 6 条评论 3 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

🍏 macos 🐛 bug 👀 requires attention 📦 package: postgres
主要语言
Python
星标
2.3k
派生
386
平均合并
4 小时 40 分钟
30 天内合并 PR
1

描述

In our current setup we have some Python code running on a docker container which uses testcontainers to create a PostgreSQL database container we can test with. All necessary requirements to start a new container from a docker image are met, and the host machine's docker client is used to start the new container. When looking at the DB container, it starts up successfully, we can access it from localhost with the port assigned to it when exposing and with the container's IP with the port we wanted to expose (5432). The problem comes if we try to access it with the Gateway IP of the network the 2 containers are in, and the port assigned to it when exposing (which is what testcontainers is doing as per my understanding).

This last part only fails when the host the 2 containers are started on is a MacOS machine, it works fine on Linux. To replicate the issue the following script can be run on a docker container that is started on a MacOS machine (with the host's docker client mounted up as per the requirements in this repo's README):

from testcontainers.postgres import PostgresContainer
import sqlalchemy

with PostgresContainer("postgres:9.6") as postgres:
    e = sqlalchemy.create_engine(postgres.get_connection_url())
    print(e.execute("select version()"))

We ran with Python 3.6 and newest versions of testcontainers, sqlalchemy and psycopg2. The above script should fail in a container started on Mac and work in a container started on Linux.

This is because the networking implementation is different between Linux and Mac. When exposing ports on Mac without providing a binding port (so docker finds a port for itself), the HostConfig/PortBindings for the newly started container gets set to "0". In contrast, it is left as an empty String on Linux. The "0" value causes the issue. We've opened a ticket about this on the docker for mac repo which has more info about this particular issue: https://github.com/docker/for-mac/issues/5588

Currently the work around we use in our code is to explicitly bind ports instead of just exposing them like so:

from testcontainers.postgres import PostgresContainer
import sqlalchemy

with PostgresContainer("postgres:9.6").with_bind_ports(5432, 47000) as postgres:
    e = sqlalchemy.create_engine(postgres.get_connection_url())
    print(e.execute("select version()"))

This version works on both Mac and Linux as the host because HostConfig/PortBindings gets updated with the port we've provided on containers started on both Mac and Linux. The downside is we always have to manually bind and find a port that is available.

The suggestion in the issue that I linked above is that for portability, the internal DNS and IP of containers should be used for communicating between containers rather than the Gateway IP. Would it make sense to make the change in testcontainers so it would use those to access the DB container rather than the Gateway IP? What was the specific reason for which Gateway IP was chosen as the preferred method of communication between containers?

I am happy to have a go at making the change in this repo if you accept contributions.

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 PostgresContainer 的入口点 get_connection_url 和 with_bind_ports 开始,然后在 Mac 和 Linux 上的容器中运行提供的 SQLAlchemy 复现代码。跟踪数据库容器地址和暴露端口的选择方式;当查询无需手动选择主机端口即可连接,并且在两个主机上的行为一致时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
docker, postgresql, python, sqlalchemy
领域
databases, devops, networking
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
42/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。