Azure / Azure/azure-sdk-for-python
Allow custom `owner_id` in `azure-eventhub` EventProcessor's `receive`/`receive_batch`
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
**Is your feature request related to a problem? Please describe.**
Currently, the `EventProcessor` in [`azure-eventhub`](https://github.com/Azure/azure-sdk-for-python/tree/d4c265e327f9b0a8e5233f6b4918f7f68b451022/sdk/eventhub/azure-eventhub) [auto-generates an `owner_id`](https://github.com/Azure/azure-sdk-for-python/blob/d4c265e327f9b0a8e5233f6b4918f7f68b451022/sdk/eventhub/azure-eventhub/azure/eventhub/_eventprocessor/event_processor.py#L115) using `str(uuid.uuid4())` when calling `receive`/`receive_batch`. This becomes problematic in environments with frequent pod/worker restarts (e.g., K8s), as each restart results in a new `owner_id`, forcing unnecessary partition rebalancing and checkpoint contention.
**Describe the solution you'd like**
Add an optional parameter (`owner_id: Optional[str] = None`) to the said methods. If provided, this value would override the auto-generated UUID. This allows users to pass a persistent identifier (e.g., pod FQDN) to:
* Reduce partition ownership juggling during brief restarts.
* Improve traceability with meaningful IDs.
Example usage:
```python
client.receive_batch(
on_event_batch=on_event_batch,
on_error=on_error,
owner_id=socket.getfqdn(),
)
```
**Describe alternatives you've considered**
As for now, I monkey patch `EventProcessor`.
**Additional context**
In my deployment, workers restart hourly (due to a custom proxy requiring credentials refresh) and use a custom MongoDB ~~(is Web scale)~~ checkpoint store with the following behavior:
* Graceful shutdowns do not release partitions.
* On restart, the same worker reclaims its previous partitions. But it only works if the `owner_id` is stable.
Explicit control over `owner_id` would greatly help in this solution.
Also, as I understand it, the Java SDK [allows you to explicitly specify `owner_id`](https://github.com/Azure/azure-sdk-for-java/blob/a10b46f9d166c8f9400ab6403e6abae07d01391b/sdk/eventhubs/azure-messaging-eventhubs/src/main/java/com/azure/messaging/eventhubs/EventHubClientBuilder.java#L1025).
I’d be happy to implement this myself if the feature is approved.
Thank you in advance.
Contributor guide
Assessment
This issue has not been assessed yet.