microsoft / microsoft/aspire

Producer fails to connect to broker on custom port 53377 connection closed by peer POLLHUP

Open
#14,445 1 comment 0 reactions 0 assignees View on GitHub
area-integrations kafka
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

Description
When starting the producer against localhost:53377 the client immediately sends an ApiVersionRequest and the broker closes the TCP connection. The producer logs repeatedly show APIVERSION_QUERY followed by Disconnected: connection closed by peer: POLLHUP and the client marks the single broker as down. This prevents any messages from being produced.

Reproduction Steps
- Configure producer bootstrap server to localhost:53377.
- Start the producer.
- Observe producer logs during initial handshake.

Actual Behavior
Producer fails to complete the initial handshake and reports the broker as down. Repeated log lines:
%6|1770824292.106|FAIL|rdkafka#producer-1| [thrd:localhost:53377/bootstrap]: localhost:53377/bootstrap: Disconnected: connection closed by peer: POLLHUP (after 6ms in state APIVERSION_QUERY)
%3|1770824292.108|ERROR|rdkafka#producer-1| [thrd:localhost:53377/bootstrap]: 1/1 brokers are down
%6|1770824292.413|FAIL|rdkafka#producer-1| [thrd:localhost:53377/bootstrap]: localhost:53377/bootstrap: Disconnected: connection closed by peer: POLLHUP (after 1ms in state APIVERSION_QUERY, 1 identical error(s) suppressed)
%3|1770824292.414|ERROR|rdkafka#producer-1| [thrd:localhost:53377/bootstrap]: 1/1 brokers are down
%3|1770824292.726|ERROR|rdkafka#producer-1| [thrd:localhost:53377/bootstrap]: 1/1 brokers are down
%3|1770824294.053|ERROR|rdkafka#producer-1| [thrd:localhost:53377/bootstrap]: 1/1 brokers are down
%3|1770824298.353|ERROR|rdkafka#producer-1| [thrd:localhost:53377/bootstrap]: 1/1 brokers are down



Expected Behavior
Producer should complete the ApiVersion handshake and establish a stable connection to the broker so messages can be produced.

Environment
Aspire version:
Client library: librdkafka / Confluent.Kafka producer (specify version)
Broker: Kafka (specify version)
Deployment: local / Docker / Kubernetes (specify)
Bootstrap server used: localhost:53377

Possible Root Causes
- No Kafka broker listening on port 53377 — a non-Kafka process or nothing is bound to that port, so the connection is accepted then closed when Kafka protocol data arrives.
- Broker misconfiguration — Kafka not configured to listen on the port the client is using or advertised listeners mismatch.
- Security mismatch — broker expects SSL/SASL while client uses PLAINTEXT, causing immediate disconnect.
- Port forwarding or proxy — an intermediary is closing the connection during handshake.

Suggested Troubleshooting Steps
- Verify what process is listening on port 53377 using netstat or ss.
- Confirm Kafka broker is running and listening on the expected port.
- Check Kafka listeners and advertised.listeners configuration for mismatches.
- Test raw TCP connection with telnet localhost 53377 to see whether the socket closes immediately.
- Ensure client security settings match broker security (PLAINTEXT vs SSL/SASL).
- If using Docker, verify container port mappings and network mode.

Temporary Workaround
Point the producer to a known working Kafka port (for example localhost:9092) if available, until the broker configuration or port binding is corrected.

Attachments and Placeholders
Please attach or paste the following into the issue to help triage:
- Broker configuration file (server.properties) or relevant docker-compose.yml.
- Broker logs around the same timestamps.
- Producer configuration snippet showing BootstrapServers and security settings.
- Environment details including OS, Docker version, and network setup.

Impact
Producer cannot connect; message production is blocked and the system reports the broker as down. This affects all features that rely on Kafka messaging.

Recommended Fix
- Ensure Kafka is bound to the port the client uses or update the client to use the broker’s actual bootstrap port.
- Align security protocols between client and broker.
- If the port was intentionally changed, update listeners and advertised.listeners so the broker advertises the correct address and port.

Notes
Logs indicate the disconnect occurs during APIVERSION_QUERY, which strongly suggests the peer is not responding to Kafka protocol messages.

### Expected Behavior

_No response_

### Steps To Reproduce

_No response_

### Exceptions (if any)

_No response_

### .NET Version info

_No response_

### Anything else?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.