Azure / Azure/azure-sdk-for-rust

[Event Hubs] Live test second_processor_displaces_first_with_consumer_disconnected fails on main

Open
#5,100 0 comments 1 reaction 1 assignee Claimed by @j7nw4r View on GitHub
Client Event Hubs
Dominant language
Rust
Stars
884
Forks
365
Avg merge
2d 19h
Merged PRs (30d)
109

Description

## Summary

The live test `second_processor_displaces_first_with_consumer_disconnected` fails against a real namespace. No partition client reports an error within its 90 second budget, so the expected `ConsumerDisconnected` never arrives.

## Motivation

The test is the only live coverage of owner level displacement through the event processor, which is what partition ownership depends on. It fails on an unmodified `main`, so the failure is in the test or in the displacement path itself, and not in any pending change. A live suite that cannot pass hides later regressions in the same area, because a reader treats the failure as known.

## Proposal

- Reproduce the failure on a namespace whose event hub has 5 partitions. The failure message reports `at least one of 5 partitions`, while `sdk/eventhubs/test-resources.bicep` provisions 4, so a partition count the test does not expect is the first thing to rule out.
- Decide whether the displacement path stopped raising `ConsumerDisconnected` or the test stopped observing it, and fix the side that is wrong.
- Keep the assertion on `ConsumerDisconnected`, because a weaker assertion would pass without proving displacement.

### Reproduction

```sh
git worktree add --detach /tmp/main-baseline origin/main
cd /tmp/main-baseline
AZURE_TEST_MODE=live EVENTHUBS_HOST=.servicebus.windows.net EVENTHUB_NAME= \
cargo test --package azure_messaging_eventhubs --test eventhubs_processor \
second_processor_displaces_first_with_consumer_disconnected -- --exact --test-threads=1
```

## Root cause, proven live on 2026-08-20

The test needs two independent fixes, and neither alone is enough. Measured against a real namespace:

| | test as on main | test fixed by #5105 |
|---|---|---|
| `azure_core_amqp` 1.1.0 from crates.io | FAIL | FAIL |
| in-tree `azure_core_amqp` 1.2.0-beta.1 | FAIL | PASS |

**Cause one, the dependency.** `azure_messaging_eventhubs` takes `azure_core_amqp.workspace = true`, and the root `Cargo.toml` pins `version = "1.1.0"` with no `path`, so the build links the crates.io copy. In that copy `src/fe2o3/receiver.rs:68-69` calls `.properties()` and then `.name()`. In `fe2o3-amqp` 0.14.0 the `name()` builder method is a typestate transition that rebuilds the struct with `properties: Default::default()` (`src/link/builder.rs:198`), so it discards them. Event Hubs carries the owner level as the `com.microsoft:epoch` link property, so the Attach frame goes out with no epoch, the broker never arbitrates, and no receiver is ever displaced. The in-tree copy fixed the order in #4805 and is version 1.2.0-beta.1, which no release carries yet.

The consequence reaches past this test: on the published dependency, `OpenReceiverOptions::owner_level` has no effect at all.

**Cause two, the test.** Even with the epoch reaching the broker, the test never made processor B attach a receiver, so no partition ever had two readers. See #5105.

## Proposal, revised

- Land #5105 for the test half.
- Flip `azure_messaging_eventhubs` and `azure_messaging_eventhubs_checkpointstore_blob` onto the in-tree `azure_core_amqp` with a `path` and a version, or release `azure_core_amqp` 1.2.0 and raise the pin. PR #5078 already carries that flip, so merging it also closes this half.
- Do not merge #5105 alone and expect the live test to pass. It will not.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.