Azure / Azure/azure-sdk-for-python

Allow custom `owner_id` in `azure-eventhub` EventProcessor's `receive`/`receive_batch`

Offen
#40,949 2 Kommentare 1 Reaktion 3 zugewiesene Personen Beansprucht von @l0lawrence Auf GitHub ansehen
Client customer-reported Event Hubs feature-request Messaging needs-team-attention
Vorherrschende Sprache
Python
Sterne
5.6k
Forks
3.4k
Ø Merge
2 T.
Gemergte PRs (30 T.)
217

Beschreibung

**Is your feature request related to a problem? Please describe.**
Currently, the `EventProcessor` in [`azure-eventhub`](https://github.com/Azure/azure-sdk-for-python/tree/d4c265e327f9b0a8e5233f6b4918f7f68b451022/sdk/eventhub/azure-eventhub) [auto-generates an `owner_id`](https://github.com/Azure/azure-sdk-for-python/blob/d4c265e327f9b0a8e5233f6b4918f7f68b451022/sdk/eventhub/azure-eventhub/azure/eventhub/_eventprocessor/event_processor.py#L115) using `str(uuid.uuid4())` when calling `receive`/`receive_batch`. This becomes problematic in environments with frequent pod/worker restarts (e.g., K8s), as each restart results in a new `owner_id`, forcing unnecessary partition rebalancing and checkpoint contention.

**Describe the solution you'd like**
Add an optional parameter (`owner_id: Optional[str] = None`) to the said methods. If provided, this value would override the auto-generated UUID. This allows users to pass a persistent identifier (e.g., pod FQDN) to:

* Reduce partition ownership juggling during brief restarts.
* Improve traceability with meaningful IDs.

Example usage:
```python
client.receive_batch(
on_event_batch=on_event_batch,
on_error=on_error,
owner_id=socket.getfqdn(),
)
```

**Describe alternatives you've considered**
As for now, I monkey patch `EventProcessor`.

**Additional context**
In my deployment, workers restart hourly (due to a custom proxy requiring credentials refresh) and use a custom MongoDB ~~(is Web scale)~~ checkpoint store with the following behavior:

* Graceful shutdowns do not release partitions.
* On restart, the same worker reclaims its previous partitions. But it only works if the `owner_id` is stable.

Explicit control over `owner_id` would greatly help in this solution.

Also, as I understand it, the Java SDK [allows you to explicitly specify `owner_id`](https://github.com/Azure/azure-sdk-for-java/blob/a10b46f9d166c8f9400ab6403e6abae07d01391b/sdk/eventhubs/azure-messaging-eventhubs/src/main/java/com/azure/messaging/eventhubs/EventHubClientBuilder.java#L1025).

I’d be happy to implement this myself if the feature is approved.

Thank you in advance.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.