Azure / Azure/azure-sdk-for-python

Allow custom `owner_id` in `azure-eventhub` EventProcessor's `receive`/`receive_batch`

Abierto
#40,949 2 comentarios 1 reacción 3 asignados Reclamado por @l0lawrence Ver en GitHub
Client customer-reported Event Hubs feature-request Messaging needs-team-attention
Lenguaje dominante
Python
Estrellas
5.6k
Forks
3.4k
Merge medio
2 d 2 h
PR fusionados (30 d)
213

Descripción

**Is your feature request related to a problem? Please describe.**
Currently, the `EventProcessor` in [`azure-eventhub`](https://github.com/Azure/azure-sdk-for-python/tree/d4c265e327f9b0a8e5233f6b4918f7f68b451022/sdk/eventhub/azure-eventhub) [auto-generates an `owner_id`](https://github.com/Azure/azure-sdk-for-python/blob/d4c265e327f9b0a8e5233f6b4918f7f68b451022/sdk/eventhub/azure-eventhub/azure/eventhub/_eventprocessor/event_processor.py#L115) using `str(uuid.uuid4())` when calling `receive`/`receive_batch`. This becomes problematic in environments with frequent pod/worker restarts (e.g., K8s), as each restart results in a new `owner_id`, forcing unnecessary partition rebalancing and checkpoint contention.

**Describe the solution you'd like**
Add an optional parameter (`owner_id: Optional[str] = None`) to the said methods. If provided, this value would override the auto-generated UUID. This allows users to pass a persistent identifier (e.g., pod FQDN) to:

* Reduce partition ownership juggling during brief restarts.
* Improve traceability with meaningful IDs.

Example usage:
```python
client.receive_batch(
on_event_batch=on_event_batch,
on_error=on_error,
owner_id=socket.getfqdn(),
)
```

**Describe alternatives you've considered**
As for now, I monkey patch `EventProcessor`.

**Additional context**
In my deployment, workers restart hourly (due to a custom proxy requiring credentials refresh) and use a custom MongoDB ~~(is Web scale)~~ checkpoint store with the following behavior:

* Graceful shutdowns do not release partitions.
* On restart, the same worker reclaims its previous partitions. But it only works if the `owner_id` is stable.

Explicit control over `owner_id` would greatly help in this solution.

Also, as I understand it, the Java SDK [allows you to explicitly specify `owner_id`](https://github.com/Azure/azure-sdk-for-java/blob/a10b46f9d166c8f9400ab6403e6abae07d01391b/sdk/eventhubs/azure-messaging-eventhubs/src/main/java/com/azure/messaging/eventhubs/EventHubClientBuilder.java#L1025).

I’d be happy to implement this myself if the feature is approved.

Thank you in advance.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.