aws / aws/aws-sdk-java-v2

BedrockRuntimeAsyncClient.invokeModelWithResponseStream swallowed the exception that being throw by Visitor

Open
#4,929 4 comments 0 reactions 0 assignees View on GitHub
bug p2
Dominant language
Java
Stars
2.6k
Forks
1k
Avg merge
2d 9h
Merged PRs (30d)
51

Description

### Describe the bug

Issue: the exception that being throw within the `Visitor` is being swallowed by the `BedrockRuntimeAsyncClient.invokeModelWithResponseStream`

### Expected Behavior

The exception should be throw by `CompletableFuture` and wrapped by `CompletionException`

### Current Behavior

The reply from the base model is being cut while the main process is closed with exit code `0` which misleading for our use case.

### Reproduction Steps

Code Snippet:
```kotlin
internal object BedrockRuntime {
...
fun invokeModelWithResponseStream(modelId: String, body: Any, type: Class, consumer: Consumer) {
val request = InvokeModelWithResponseStreamRequest.builder()
.modelId(modelId)
.body(SdkBytes.fromUtf8String(objectMapper.writeValueAsString(body)))
.contentType(JSON_MEDIA_TYPE)
.accept(JSON_MEDIA_TYPE)
.build()
val visitor = InvokeModelWithResponseStreamResponseHandler.Visitor.builder()
.onChunk {
val payload = it
.bytes()
.asUtf8String()
val response = objectMapper.readValue(payload, type)
consumer.accept(response)
}
.build()
val handler = InvokeModelWithResponseStreamResponseHandler.builder()
.subscriber(visitor)
.build()
try {
asyncClient.invokeModelWithResponseStream(request, handler)
.join()
} catch (exception: Exception) {
exception.printStackTrace()
}
}
...
}
...
object Application {
@JvmStatic
fun main(args: Array) {
...
var counter = 0
BedrockRuntime.invokeModelWithResponseStream(CLAUDE_MODEL_ID, request, ClaudeBodyResponse::class.java) {
counter++
print(it.completion)
if (counter > 10) {
throw RuntimeException("TEST")
}
}
...
}
}
```

Output:
```
Berikut adalah beberapa tips unt
Process finished with exit code 0
```

### Possible Solution

_No response_

### Additional Information/Context

Kotlin Version: 1.9.22

### AWS Java SDK version used

2.24.1

### JDK version used

openjdk version "21.0.2" 2024-01-16 LTS OpenJDK Runtime Environment Temurin-21.0.2+13 (build 21.0.2+13-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.2+13 (build 21.0.2+13-LTS, mixed mode)

### Operating System and version

macOS Sonoma Version 14.3.1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.