aws / aws/aws-sdk-cpp

InvokeModelWithBidirectionalStreamAsync() can block the calling thread forever when the peer never sends any response - no timeout at any layer

Offen
#3,923 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
C++
Sterne
2.2k
Forks
1.2k
Ø Merge
4 T. 11 Std.
Gemergte PRs (30 T.)
13

Beschreibung

## Summary

`BedrockRuntimeClient::InvokeModelWithBidirectionalStreamAsync()` is documented and named as an asynchronous method, but part of the initial event-stream write work runs **synchronously on the calling thread**, inside the call itself, with **no timeout anywhere in the chain** down to raw HTTP/2 flow control. If the server accepts the connection/stream but never sends any response (and therefore never sends a `WINDOW_UPDATE` frame either), the calling thread can hang forever inside the "async" call, not merely leave a callback unfired.

## Environment

- aws-sdk-cpp version: **1.11.890** (built with the CRT HTTP client, `AWS_SDK_USE_CRT_HTTP` / push-based `WriteData` path)
- Service: Bedrock Runtime, `InvokeModelWithBidirectionalStreamAsync` (used for Nova Sonic real-time voice streaming)
- Platform: Windows, MSVC, x86/Win32
- Underlying CRT: aws-crt-cpp (commit pinned by aws-sdk-cpp 1.11.890) / aws-c-http (commit pinned by that aws-crt-cpp)

## Repro

1. Stand up a local HTTP/2 (TLS, self-signed cert is fine) server that:
- Accepts the TCP/TLS/HTTP2 connection and the incoming stream normally.
- Never sends any `WINDOW_UPDATE` frame beyond the connection/stream defaults, and never sends any response (no headers, no data).
2. Call `BedrockRuntimeClient::InvokeModelWithBidirectionalStreamAsync()` against it (dummy credentials are fine - the server never needs to validate auth), with a `streamReadyHandler` that sends a normal small sequence of bidirectional-streaming events (e.g. a Nova Sonic `sessionStart`/`promptStart`/`contentStart`/`textInput`/`contentEnd`/`promptEnd`/`sessionEnd` sequence) and then calls `Close()` on the input stream.
3. Place a diagnostic statement immediately after the call to `InvokeModelWithBidirectionalStreamAsync()` in your own code.

**Expected**: the call returns promptly (it's documented/named as async), and only the response-received/outcome callback is left waiting - ideally itself bounded by some timeout, or at least clearly the caller's own responsibility to bound since it's callback-based.

**Actual**: the diagnostic statement never executes. The calling thread is parked indefinitely (confirmed via process inspection: near-zero CPU, threads alive, no progress even after 9+ minutes in one run) inside the call itself.

## Root cause (traced through the actual 1.11.890 source)

1. `generated/src/aws-cpp-sdk-bedrock-runtime/source/BedrockRuntimeClient.cpp` (~L401-427), the CRT/push-based branch of `InvokeModelWithBidirectionalStreamAsync()`:
```cpp
auto asyncTask = smithy::client::CreateSmithyBidirectionalWriteDataTask(
this, requestCopy, handler, handlerContext, eventEncoderStream, writeDataStreamBuf, std::move(endpointCallback),
std::move(authCallback));
auto sem = asyncTask.GetSemaphore();
m_clientConfiguration.executor->Submit(std::move(asyncTask));
sem->WaitOne();
streamReadyHandler(*eventEncoderStream);
```
`sem->WaitOne()` is unconditional (no timeout). Once it returns (i.e. once the background task's `HttpWriteDataStreamBuf::Initialize()` succeeds), `streamReadyHandler` - the caller-supplied write callback - is invoked **directly on the calling thread**, as the last statement of this method. There is nothing else in the function body after this call; the method returns only once `streamReadyHandler` returns.

2. Inside `streamReadyHandler`, calling `Close()` on the event-stream input (`Model::InvokeModelWithBidirectionalStreamInput` → inherited `Aws::Utils::Event::EventEncoderStream::Close()`) is a pure passthrough down to `HttpWriteDataStreamBuf::Close()` → `SendBuffer(endStream=true)`. The same `SendBuffer()` is used for every write (`overflow()`, `xsputn()`, `sync()`, and `Close()`).

3. `src/aws-cpp-sdk-core/source/utils/stream/HttpWriteDataStreamBuf.cpp` (~L148-181), `SendBuffer()`:
```cpp
m_stream->WriteData(data, [this](int errorCode) -> void {
std::unique_lock const lock{m_writeMutex};
m_writeInProgress = false;
m_writeError = (errorCode != AWS_ERROR_SUCCESS);
m_writeComplete.notify_one();
}, endStream);

std::unique_lock lock{m_writeMutex};
m_writeComplete.wait(lock, [this]() -> bool { return !m_writeInProgress; });
```
This wait has **no timeout parameter and no timed overload anywhere in the class**.

4. The completion callback only fires once aws-c-http's `aws_h2_stream_encode_data_frame()` (`source/h2_stream.c`, ~L829-846) actually encodes the write into an outgoing DATA frame:
```c
if (stream->thread_data.window_size_peer <= AWS_H2_MIN_WINDOW_SIZE) {
/* The stream is stalled now */
*data_encode_status = AWS_H2_DATA_ENCODE_ONGOING_WINDOW_STALLED;
return AWS_OP_SUCCESS;
}
```
If the stream is window-stalled, this returns having done **nothing** - no error, no retry scheduling, no deadline. The queued write just sits there.

5. `window_size_peer` is this side's outbound HTTP/2 flow-control credit as granted by the peer - seeded from the RFC 7540 §6.5.2 default (65535 bytes) at connection/stream setup (`source/h2_connection.c` ~L387-388, `source/h2_stream.c` ~L741-745), and only ever replenished by a `WINDOW_UPDATE` frame **from the peer**. A peer that never sends anything never sends one either. Once cumulative outstanding bytes across the whole stream exceed that window, every subsequent write - including, in this repro, the one inside `Close()` - stalls forever.

6. The one native mechanism that exists specifically for "peer never responds" is `response_first_byte_timeout_ms` (`aws-c-http`'s `aws_http_connection_manager_options` / `aws_http_make_request_options`), but per its own doc comment it is explicitly HTTP/1.1-only:
```c
/**
* ...
* TODO: Only supported in HTTP/1.1 now, support it in HTTP/2
*/
uint64_t response_first_byte_timeout_ms;
```
confirmed by grepping the whole `aws-c-http` tree: it's referenced only in `h1_connection.c`/`h1_stream.c`, never in `h2_connection.c`/`h2_stream.c`. It is also not exposed anywhere through `aws-crt-cpp`'s or `aws-sdk-cpp`'s C++ wrapper types, even for HTTP/1.1.

7. Separately, `ClientConfiguration::requestTimeoutMs` exists and is checked in the SDK's synchronous `MakeRequest()` response wait (`src/aws-cpp-sdk-core/source/http/crt/CRTHttpClient.cpp` ~L537-559), but is never referenced anywhere in the connection-acquisition (`AcquireConnection`, ~L721-745) or bidirectional-streaming write path - so it provides no protection here even when configured.

## Impact

For a caller of `InvokeModelWithBidirectionalStreamAsync()`, there is currently no supported way, at any layer, to bound how long the call itself (not just the eventual response callback) can block against a peer that accepts the stream but never communicates further. This is worse than "the outcome callback never fires" - the calling application's own thread is the one that hangs, which for an application built expecting an async, callback-driven API (e.g. a call-handling engine dispatching this from a request-processing thread) can freeze application logic entirely, not just leave a background task dangling.

## Suggested areas to address (not prescribing the fix, just where the gaps are)

- Expose `response_first_byte_timeout_ms` (or an HTTP/2-appropriate equivalent) through `aws-crt-cpp` and `aws-sdk-cpp`'s `ClientConfiguration`, and implement it for HTTP/2 in `aws-c-http` (currently HTTP/1.1-only by an existing TODO).
- At minimum, give `HttpWriteDataStreamBuf::SendBuffer()`'s write-completion wait a timeout path, so a stalled write surfaces as an error to the caller instead of blocking forever.
- Consider whether `streamReadyHandler` needs to run synchronously on the calling thread inside `InvokeModelWithBidirectionalStreamAsync()` at all - if the intent is a genuinely async method, dispatching this initial write burst through the executor (like the rest of the task) rather than back on the caller's thread would preserve the documented async contract even without a new timeout feature.

## Related, but distinct issues (searched, not duplicates)

- #3917 / #3911 - `Aws::ShutdownAPI()` hang after a stream has already *completed* (self-reference cycle pinning a connection) - already fixed in PR #3919 (1.11.890). This report is a different bug: the hang happens *during* the initial call, before any response, triggered by HTTP/2 flow control against a non-responsive peer, not by a resource cycle at shutdown.
- #3649, #3650 - connectivity/error-visibility issues, not hangs of this kind.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne mit generated/src/aws-cpp-sdk-bedrock-runtime/source/BedrockRuntimeClient.cpp und src/aws-cpp-sdk-core/source/utils/stream/HttpWriteDataStreamBuf.cpp und verfolge dann den CRT HTTP/2-Schreibpfad durch aws-c-http's source/h2_stream.c. Reproduziere das Szenario mit dem ins Stocken geratenen Peer und untersuche die bestehende Timeout-Behandlung. Als erledigt gilt, dass der asynchrone Aufruf und seine anfänglichen Schreibvorgänge zeitlich begrenzt sind und der Fehler beobachtbar ist, statt auf unbestimmte Zeit hängen zu bleiben.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
aws, cpp
Bereich
api, backend, networking
Issue-Typ
Bug
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Aktiv
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
38/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.