BedrockRuntimeAsyncClient.invokeModelWithResponseStream swallowed the exception that being throw by Visitor
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 51
Description
### Describe the bug
Issue: the exception that being throw within the `Visitor` is being swallowed by the `BedrockRuntimeAsyncClient.invokeModelWithResponseStream`
### Expected Behavior
The exception should be throw by `CompletableFuture` and wrapped by `CompletionException`
### Current Behavior
The reply from the base model is being cut while the main process is closed with exit code `0` which misleading for our use case.
### Reproduction Steps
Code Snippet:
```kotlin
internal object BedrockRuntime {
...
fun invokeModelWithResponseStream(modelId: String, body: Any, type: Class, consumer: Consumer) {
val request = InvokeModelWithResponseStreamRequest.builder()
.modelId(modelId)
.body(SdkBytes.fromUtf8String(objectMapper.writeValueAsString(body)))
.contentType(JSON_MEDIA_TYPE)
.accept(JSON_MEDIA_TYPE)
.build()
val visitor = InvokeModelWithResponseStreamResponseHandler.Visitor.builder()
.onChunk {
val payload = it
.bytes()
.asUtf8String()
val response = objectMapper.readValue(payload, type)
consumer.accept(response)
}
.build()
val handler = InvokeModelWithResponseStreamResponseHandler.builder()
.subscriber(visitor)
.build()
try {
asyncClient.invokeModelWithResponseStream(request, handler)
.join()
} catch (exception: Exception) {
exception.printStackTrace()
}
}
...
}
...
object Application {
@JvmStatic
fun main(args: Array) {
...
var counter = 0
BedrockRuntime.invokeModelWithResponseStream(CLAUDE_MODEL_ID, request, ClaudeBodyResponse::class.java) {
counter++
print(it.completion)
if (counter > 10) {
throw RuntimeException("TEST")
}
}
...
}
}
```
Output:
```
Berikut adalah beberapa tips unt
Process finished with exit code 0
```
### Possible Solution
_No response_
### Additional Information/Context
Kotlin Version: 1.9.22
### AWS Java SDK version used
2.24.1
### JDK version used
openjdk version "21.0.2" 2024-01-16 LTS OpenJDK Runtime Environment Temurin-21.0.2+13 (build 21.0.2+13-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.2+13 (build 21.0.2+13-LTS, mixed mode)
### Operating System and version
macOS Sonoma Version 14.3.1
Contributor guide
Assessment
This issue has not been assessed yet.