testcontainers / testcontainers/testcontainers-java

Bad KAFKA_ADVERTISED_LISTENERS when exposing host ports to the container

Open
#3,187 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

modules/kafka
Dominant language
Java
Stars
8.7k
Forks
1.9k
Avg merge
2d 17h
Merged PRs (30d)
9

Description

This issue relates to the Kafka module.

When using the testcontainers utility to expose host ports to all containers (see https://www.testcontainers.org/features/networking/#exposing-host-ports-to-the-container), KafkaContainer faills to start Kafka due to an internal misconfiguration.

The error seen from the Kafka container logs is the following:

java.lang.IllegalArgumentException: requirement failed: Each listener must have a different port, listeners: PLAINTEXT://localhost:32889,BROKER://192.168.0.2:9092,BROKER://172.17.0.4:9092

From my analysis, it seems that, when using the feature, testcontainers adds the additional bridge network to each container which messes up with the setup of the KAFKA_ADVERTISED_LISTENERS environment variable of the KafkaContainer class (see https://github.com/testcontainers/testcontainers-java/blob/master/modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java#L115-L121).

We change the affected part to use different ports when multiple networks are present, but I'm not sure of the implications. Any thoughts on the subject?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java around lines 115-121 and review the networking documentation for exposing host ports to containers. Reproduce the failure with an additional bridge network and check how KAFKA_ADVERTISED_LISTENERS is assembled. Done means KafkaContainer starts successfully with distinct listener ports in this configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java, kafka
Domain
distributed-systems, networking, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.