Azure / Azure/azure-sdk-for-python

Allow custom `owner_id` in `azure-eventhub` EventProcessor's `receive`/`receive_batch`

Ouverte
#40,949 2 commentaires 1 réaction 3 personnes assignées Réclamée par @l0lawrence Voir sur GitHub
Client customer-reported Event Hubs feature-request Messaging needs-team-attention
Langage dominant
Python
Étoiles
5.6k
Forks
3.4k
Merge moyen
2 j 2 h
PR mergées (30 j)
213

Description

**Is your feature request related to a problem? Please describe.**
Currently, the `EventProcessor` in [`azure-eventhub`](https://github.com/Azure/azure-sdk-for-python/tree/d4c265e327f9b0a8e5233f6b4918f7f68b451022/sdk/eventhub/azure-eventhub) [auto-generates an `owner_id`](https://github.com/Azure/azure-sdk-for-python/blob/d4c265e327f9b0a8e5233f6b4918f7f68b451022/sdk/eventhub/azure-eventhub/azure/eventhub/_eventprocessor/event_processor.py#L115) using `str(uuid.uuid4())` when calling `receive`/`receive_batch`. This becomes problematic in environments with frequent pod/worker restarts (e.g., K8s), as each restart results in a new `owner_id`, forcing unnecessary partition rebalancing and checkpoint contention.

**Describe the solution you'd like**
Add an optional parameter (`owner_id: Optional[str] = None`) to the said methods. If provided, this value would override the auto-generated UUID. This allows users to pass a persistent identifier (e.g., pod FQDN) to:

* Reduce partition ownership juggling during brief restarts.
* Improve traceability with meaningful IDs.

Example usage:
```python
client.receive_batch(
on_event_batch=on_event_batch,
on_error=on_error,
owner_id=socket.getfqdn(),
)
```

**Describe alternatives you've considered**
As for now, I monkey patch `EventProcessor`.

**Additional context**
In my deployment, workers restart hourly (due to a custom proxy requiring credentials refresh) and use a custom MongoDB ~~(is Web scale)~~ checkpoint store with the following behavior:

* Graceful shutdowns do not release partitions.
* On restart, the same worker reclaims its previous partitions. But it only works if the `owner_id` is stable.

Explicit control over `owner_id` would greatly help in this solution.

Also, as I understand it, the Java SDK [allows you to explicitly specify `owner_id`](https://github.com/Azure/azure-sdk-for-java/blob/a10b46f9d166c8f9400ab6403e6abae07d01391b/sdk/eventhubs/azure-messaging-eventhubs/src/main/java/com/azure/messaging/eventhubs/EventHubClientBuilder.java#L1025).

I’d be happy to implement this myself if the feature is approved.

Thank you in advance.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.