testcontainers / testcontainers/testcontainers-python

Question: Cannot start KafkaContainer() (on windows(?))

Open
#1,015 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

📖 documentation
Dominant language
Python
Stars
2.3k
Forks
386
Avg merge
4h 40m
Merged PRs (30d)
1

Description

I am trying to create a KafkaContainer for my integration test, but I have issues starting it.

Using testcontainers==4.12.0, the following snippet has no issues:

from testcontainers.kafka import KafkaContainer
kafka_container = KafkaContainer()
kafka_container.start()
kafka_bootstrap_servers = cls.kafka_container.get_bootstrap_server()

However, upgrading to testcontainers==4.14.2, the same snippet gives the following issue:

Traceback (most recent call last):
  File "...\test.py", line 42, in setUpClass
    cls.kafka_container.start()
  File "...\env\Lib\site-packages\testcontainers\kafka\__init__.py", line 192, in start
    self.tc_start()
  File "...\env\Lib\site-packages\testcontainers\kafka\__init__.py", line 165, in tc_start
    port = self.get_exposed_port(self.port)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\env\Lib\site-packages\testcontainers\core\container.py", line 251, in get_exposed_port
    return self._get_exposed_port(port)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\env\Lib\site-packages\testcontainers\core\container.py", line 257, in _get_exposed_port
    return int(self.get_docker_client().port(c.id, port))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\env\Lib\site-packages\testcontainers\core\docker_client.py", line 170, in port
    raise ConnectionError(f"Port mapping for container {container_id} and port {port} is not available")
ConnectionError: Port mapping for container 06001694f7ed34dac7f89c4f3a3488d23a884338ec2cc8b367120c2452d39382 and port 9093 is not available

I did a simple investigation and found out that the waiting logic has changed between these two tags. When I use tag 13.0 and 13.1, there is a "legacy deprecation warning" but KafkaContainer can still be started.

...\test.py:40: DeprecationWarning: The wait_for_logs function with string or callable predicates is deprecated and will be removed in a future version. Use structured wait strategies instead: container.waiting_for(LogMessageWaitStrategy('ready')) or container.waiting_for(LogMessageWaitStrategy(re.compile(r'pattern')))
  wait_for_logs(cls.kafka_container, "started")

Specifically, the ConnectionError I mentioned above occurred when I upgraded 13.1 to 13.2. The changes I suspect could cause this would be this line

Not sure if this is caused by wrong usage of the library or my personal setup. It is worth noting that I have to disable Ryuk when running the code snippet (using TESTCONTAINERS_RYUK_DISABLED=true) and I am running the code snippet from Windows cmd.

Let me know if there are more information I could provide to investigate this further

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with testcontainers/kafka/init.py and testcontainers/core/docker_client.py, focusing on the waiting and port-mapping changes between v4.13.1 and v4.13.2. Reproduce the supplied KafkaContainer snippet on Windows with Ryuk disabled, then verify that the container starts and get_bootstrap_server() returns a usable endpoint.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, kafka, python
Domain
devops, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.