High Incoming request sum on Azure Event Hub
- Lingua principale
- Python
- Stelle
- 1.4k
- Fork
- 269
- Merge medio
- 1g 1h
- PR unite (30g)
- 6
Descrizione
**Describe the bug**
I was able to connect to Azure Event Hub but when I run producer and consumer and connect to a topic I see an Incoming (Sum) and Successful Request (Sum) spike to 4K and stays constantly at 4K even when no message is sent
**Expected behavior**
As there is only 1 pod running producer and consumer there should be fewer Incoming and Successful Request (Sum) because I have one more Kubernetes which is running producer and consumer written in Node.js which hardly consumes 1k Incoming and Successful Request (Sum) which has a minimum of 5 Pods running all the time
**Environment (please complete the following information):**
- aiokafka version (`python -c "import aiokafka; print(aiokafka.__version__)"`): "0.8.1"
- Kafka Broker version (`kafka-topics.sh --version`):
- Other information (Confluent Cloud version, etc.): Azure Event Hub as Kafka
**Reproducible example**
```python
# Add a short Python script or Docker configuration that can reproduce the issue.
Consumer
async def _subscribe_to_topic(self, topic: str, group_id: str):
consumer = AIOKafkaConsumer(topic,
bootstrap_servers=self.server,
group_id=group_id,
sasl_plain_username='$ConnectionString',
sasl_plain_password=self.kafkaPass,
sasl_mechanism='PLAIN',)
await consumer.start()
print("consumer started")
try:
async for msg in consumer:
payload = json.loads(msg.value)
print(payload)
try:
charging_station_id = payload.get("charging_station_id")
await self._process_payload(charging_station_id, payload)
except Exception as e:
logger.error(f"Error processing payload: {e}, payload: {payload}")
finally:
await consumer.stop()
print("consumer stopped")
Producer
async def _publish(self, topic: str, envelope: Envelope):
envelope.timestamp = datetime.now(timezone.utc)
endpoint = os.getenv("PUPSUB_KAFKA_SERVER")
kafkaPass = os.getenv("PUPSUB_KAFKA_PASSWORD")
if endpoint is None or kafkaPass is None:
raise ValueError("PUPSUB_KAFKA_ENDPOINT not set or PUPSUB_KAFKA_PASSWORD not set ")
producer = AIOKafkaProducer(bootstrap_servers=endpoint,
sasl_mechanism='PLAIN',
sasl_plain_username='$ConnectionString',
sasl_plain_password=kafkaPass,
enable_idempotence=True)
# Get cluster layout and initial topic/partition leadership information
await producer.start()
try:
# Produce message
await producer.send_and_wait(topic, envelope.json())
finally:
# Wait for all pending messages to be delivered or expire.
await producer.stop()
```

Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.