Azure / Azure/azure-sdk-for-cpp
[Event Hubs] Track GA readiness for azure-messaging-eventhubs
- Dominant language
- C++
- Stars
- 205
- Forks
- 172
- Avg merge
- 1d 15m
- Merged PRs (30d)
- 33
Description
## Summary
`azure-messaging-eventhubs` is not ready for a GA `1.0.0` release. It loses a link permanently after one detach, never refreshes the authentication token, retries errors that are not retryable, silently ignores the partition key on every batch send, and exposes unreleased `azure-core-amqp` internal types in its public headers.
This is the root tracking issue for GA readiness. It inventories the gaps against the .NET library (`Azure.Messaging.EventHubs`), which is the reference bar, and it groups the work into eight child issues.
The request: work the child issues, close every GA blocker, and meet the validation gate at the end of this issue before the team tags `1.0.0`.
Severity levels, used here and in every child issue:
- **GA blocker**: the team must not ship `1.0.0` with this item open. The item is a breaking change that cannot land after GA, a correctness or data-loss defect, or a hard release-checklist requirement.
- **GA quality bar**: the item does not block `1.0.0`. Without it, the library compares poorly to the .NET library.
- **Post GA**: the team tracks the item and can defer it.
All code anchors refer to commit `f1039fe23`. Line numbers can move; the symbol names are the stable reference.
## Motivation
The C++ client layer is a functional port of the Go client. It reproduces the operations, but not the transport-hardening layer that the .NET library encodes after years of production use.
| Gap | C++ | .NET reference |
|---|---|---|
| A faulted link is never rebuilt | `src/producer_client.cpp:196-234`, `src/partition_client.cpp:225-272` | `Amqp/AmqpProducer.cs:107,179,290` |
| The CBS token is cached with no expiry test | `azure-core-amqp/src/amqp/connection.cpp:175-183,227` | `Amqp/AmqpConnectionScope.cs:73,736` |
| The retry engine ignores its own transient classifier | `src/retry_operation.cpp:13,57,67-83`, `src/eventhubs_utilities.cpp:87-108` | `Core/BasicRetryPolicy.cs:142-189` |
| The processor loop exits silently, with no error callback | `src/processor.cpp:59-62,112-115` | `ProcessErrorAsync` |
| The batch partition key goes to the wrong AMQP section | `src/event_data_batch.cpp:33-34,73` | `Amqp/AmqpMessageConverter.cs:359,390` |
| Public headers embed unreleased `_internal` AMQP types | 5 public headers | the transport is fully internal |
The impact: any client that outlives one token lifetime (about 60 to 90 minutes for Microsoft Entra ID) or one network fault stops working and cannot recover without a process restart. A processor whose checkpoint store fails once exits its loop with only a warning log. On Windows, the blob store parses sequence numbers with `std::stol`, so any checkpoint above 2^31 is unreadable (`blob_checkpoint_store.cpp:24`).
The partition key defect is the most immediate risk, and a live test proves it. The library sends the key in the wrong AMQP section of the batch envelope, so the service round-robins the batch instead of routing it. The service reports no error, and the consumed events still report the correct partition key, so no caller can detect the fault. See #7257 for the evidence.
The GA surface is also unsettled. `azure-core-amqp` is at `1.0.0-beta.12 (Unreleased)` and is mid migration from uAMQP to a Rust implementation, and the client sources still branch on `ENABLE_UAMQP` and `ENABLE_RUST_AMQP`. Breaks of this class are only possible before `1.0.0`, which is why the surface work lands first.
## Proposal
Work the eight child issues in the four phases below. Each child carries its own evidence, its own severity tags, and its own validation gate.
The send flow and the authentication flow come first. The order below reflects that, and it also reflects which fixes depend on the AMQP transport decision. The dependency was measured rather than assumed: `src/event_data_batch.cpp` contains no `ENABLE_UAMQP` or `ENABLE_RUST_AMQP` branch, so the send path defect is transport-independent and needs no gate. `src/producer_client.cpp` contains five such branches, one of them inside `EnsureSender` at line 214, and `sdk/core/azure-core-amqp/src/amqp/connection.cpp` is written almost entirely under `#if ENABLE_UAMQP`. Link rebuild and token refresh are therefore transport-coupled, and writing them before the transport is chosen means writing them twice.
The gate is the transport decision alone, which is the first checkbox of #7253. It is a decision, not code. The remaining four items of #7253 are public surface cleanup, and they run later without blocking the reliability work.
| Phase | Work | Issue | GA blockers | GA quality bar |
|---|---|---|---|---|
| 1. Send path correctness | WS5: Fix the batch envelope partition key. Transport-independent, so it starts immediately. | #7257 | 4 | 0 |
| 2. Gate | WS1 first item only: choose the GA AMQP transport. | #7253 | 1 of 5 | 0 |
| 3. Auth and send reliability | WS2: CBS token refresh, sender link rebuild, receiver rebuild, retry classification, cancellable backoff. | #7254 | 5 | 2 |
| 4. Everything else | WS1 remaining surface cleanup | #7253 | 4 of 5 | 0 |
| 4. Everything else | WS3: Client lifecycle and thread safety | #7255 | 3 | 2 |
| 4. Everything else | WS4: Processor and checkpoint store | #7256 | 2 | 6 |
| 4. Everything else | WS6: Finalize the public API before GA | #7258 | 2 | 3 |
| 4. Everything else | WS7: Distributed tracing | #7259 | 2 | 0 |
| 4. Everything else | WS8: Tests, documentation, and release engineering | #7260 | 4 | 3 |
Two notes on the phases. Phase 1 and phase 2 run in parallel, because the partition key fix touches no transport-conditional code. Phase 4 is unordered, and #7260 trails the workstreams it tests.
The connection string authentication path is also part of the authentication flow, and that work is already in review. See the related work section below.
### Post GA backlog
- Buffered producer client, which is .NET-only surface today.
- Idempotent partition publishing.
- Geo-replication convenience features.
- WebSocket and proxy transport options, if #7258 defers them.
### Related work
Issue #7250 restores the connection string constructors that `1.0.0-beta.11` removed. It is a developer ergonomics restoration rather than a GA blocker, so it is not a child of this issue. #7260 must coordinate its README rewrite with that work, so the examples match the surface that ships.
### Resolved severity calls
Three severity calls were open when this issue was filed. The Azure SDK guidelines and the four other language libraries have since been read directly, and all three are now decided. Each child issue records its own evidence.
- **Distributed tracing (#7259) is now a GA blocker, raised from quality bar.** The C++ implementation guidelines are explicit: "DO create a new trace span for each API call" [`cpp-tracing-span-per-call`], alongside the general "DO support OpenTelemetry for distributed tracing" [`general-tracing-opentelemetry`]. The earlier tag assumed the C++ position was unstated. It is not. Repository practice is the counterweight: of the C++ client packages, only `azure-security-attestation` creates spans, and `azure-storage-blobs`, `azure-security-keyvault-secrets`, and `azure-identity` have shipped stable releases without tracing. A recorded architecture board waiver is the only path that lowers the tag.
- **The error reason enum (#7258) stays a GA quality bar, and the reason is now verified.** `cpp-versioning-backwards-compatibility` constrains source compatibility, and the C++ guidelines record no ABI promise, so adding a reason enum plus an accessor to `EventHubsException` after `1.0.0` keeps every caller compiling. The tag holds only for a supplement. Replacing the existing condition string breaks callers, so #7258 now carries the supplement-or-replace decision as a separate blocker.
- **The processor defaults (#7256) are documentation work, not alignment work.** The five libraries sit in two camps: .NET, Java, and Python use Greedy with a 30 second interval; Go and C++ use Balanced with a 10 second interval. C++ matches Go exactly, which is the library it was ported from, so there is no cross-language parity to align to. The item is no longer a candidate for promotion. The comparison did surface a real gap that the earlier analysis missed: Go and Python both jitter the load balancing interval and C++ does not, so C++ processors that start together stay in lockstep. #7256 now carries that as its own item.
The Azure SDK design guideline pages have now been read. The earlier caveat, that no repository document forbids the shapes proposed here, is withdrawn and replaced by the citations above.
## Validation
The team tags `1.0.0` only after every item below passes.
- [ ] Every GA blocker item across the eight child issues is closed.
- [ ] A producer runs 24 hours against a live Event Hub with induced link detaches, and it recovers each time without a process restart.
- [ ] A processor runs 24 hours against a live Event Hub with induced link detaches and one induced checkpoint store outage, and it keeps consuming and reports the errors through its error callback.
- [ ] A client runs past two token lifetimes and continues to send and to receive.
- [ ] Checkpoints written by the .NET processor are read correctly by the C++ processor on Windows and on Linux, including a sequence number above 2^31.
- [ ] A batch sent with a partition key lands on exactly one partition, and each event carries the correct `x-opt-partition-key` value.
- [ ] A produced event carries W3C trace context and a consumer continues the same trace, or the architecture board waiver for tracing is recorded in #7259.
- [ ] The architecture board API sign-off and the API surface artifact are archived.
- [ ] CI is green on the full platform matrix, including the live tests and the vcpkg port builds.
Contributor guide
Assessment
This issue has not been assessed yet.