open-telemetry / open-telemetry/opentelemetry-python-contrib

[ConfluentKafkaInstrumentor] Not all methods are supported by returned proxy objects

Open
#4,278 2 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
1.1k
Forks
1.1k
Avg merge
4d 15h
Merged PRs (30d)
16

Description

Describe your environment
  • Python: 3.9.25
  • OS: macOS Darwin 25.3.0 (also reproducible on Linux)
  • confluent-kafka: 2.6.1
  • opentelemetry-instrumentation-confluent-kafka: 0.60b1
  • opentelemetry-api: 1.29.0
  • opentelemetry-sdk: 1.29.0
What happened?

Calling list_topics() or assignment() on a ProxiedProducer or ProxiedConsumer returned by ConfluentKafkaInstrumentor().instrument_producer() / instrument_consumer() causes a segmentation fault or error RuntimeError: Consumer closed.

Reproducer:

from confluent_kafka import Producer
from opentelemetry.instrumentation.confluent_kafka import ConfluentKafkaInstrumentor

# Create and instrument producer
p = Producer({'bootstrap.servers': 'localhost:9092'})
p_instrumented = ConfluentKafkaInstrumentor().instrument_producer(p)

# This causes a segfault
topics = p_instrumented.list_topics(timeout=5)

Note:
There may be more methods missing in the proxy object. And I propose that the Proxy class should proxy all method calls on the actual confluent Kafka client when not defined in the proxy class.

Steps to Reproduce

Start kafka using docker and run following code:

from confluent_kafka import Producer
from opentelemetry.instrumentation.confluent_kafka import ConfluentKafkaInstrumentor

# Create and instrument producer
p = Producer({'bootstrap.servers': 'localhost:9092'})
p_instrumented = ConfluentKafkaInstrumentor().instrument_producer(p)

# This causes a segfault
topics = p_instrumented.list_topics(timeout=5)
Expected Result

The app does not crash.

Actual Result

segmentation fault or error RuntimeError: Consumer closed when calling list_topics()

Additional context

No response

Would you like to implement a fix?

None

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ProxiedProducer and ProxiedConsumer implementations used by instrument_producer() and instrument_consumer(), then reproduce the failure with list_topics() and assignment() using the provided examples. Done means these calls no longer segfault or raise the reported closed-consumer error, with the supported proxy behavior verified against the actual Confluent Kafka clients.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka, python
Domain
backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.