Azure / Azure/azure-sdk-for-python

Allow custom `owner_id` in `azure-eventhub` EventProcessor's `receive`/`receive_batch`

Open
#40,949 2 comments 1 reaction 3 assignees Claimed by @l0lawrence View on GitHub
Client customer-reported Event Hubs feature-request Messaging needs-team-attention
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

**Is your feature request related to a problem? Please describe.**
Currently, the `EventProcessor` in [`azure-eventhub`](https://github.com/Azure/azure-sdk-for-python/tree/d4c265e327f9b0a8e5233f6b4918f7f68b451022/sdk/eventhub/azure-eventhub) [auto-generates an `owner_id`](https://github.com/Azure/azure-sdk-for-python/blob/d4c265e327f9b0a8e5233f6b4918f7f68b451022/sdk/eventhub/azure-eventhub/azure/eventhub/_eventprocessor/event_processor.py#L115) using `str(uuid.uuid4())` when calling `receive`/`receive_batch`. This becomes problematic in environments with frequent pod/worker restarts (e.g., K8s), as each restart results in a new `owner_id`, forcing unnecessary partition rebalancing and checkpoint contention.

**Describe the solution you'd like**
Add an optional parameter (`owner_id: Optional[str] = None`) to the said methods. If provided, this value would override the auto-generated UUID. This allows users to pass a persistent identifier (e.g., pod FQDN) to:

* Reduce partition ownership juggling during brief restarts.
* Improve traceability with meaningful IDs.

Example usage:
```python
client.receive_batch(
on_event_batch=on_event_batch,
on_error=on_error,
owner_id=socket.getfqdn(),
)
```

**Describe alternatives you've considered**
As for now, I monkey patch `EventProcessor`.

**Additional context**
In my deployment, workers restart hourly (due to a custom proxy requiring credentials refresh) and use a custom MongoDB ~~(is Web scale)~~ checkpoint store with the following behavior:

* Graceful shutdowns do not release partitions.
* On restart, the same worker reclaims its previous partitions. But it only works if the `owner_id` is stable.

Explicit control over `owner_id` would greatly help in this solution.

Also, as I understand it, the Java SDK [allows you to explicitly specify `owner_id`](https://github.com/Azure/azure-sdk-for-java/blob/a10b46f9d166c8f9400ab6403e6abae07d01391b/sdk/eventhubs/azure-messaging-eventhubs/src/main/java/com/azure/messaging/eventhubs/EventHubClientBuilder.java#L1025).

I’d be happy to implement this myself if the feature is approved.

Thank you in advance.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.