BedrockAgentCore: invoke_agent_runtime buffers entire response instead of streaming in real-time
- Dominant language
- Ruby
- Stars
- 3.7k
- Forks
- 1.2k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 5
Description
### Describe the bug
The `invoke_agent_runtime` method buffers the entire EventStream response before yielding any chunks to `resp.response.each`, completely defeating the purpose of streaming. All chunks arrive simultaneously after the full response is generated, making real-time streaming UX impossible.
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Expected Behavior
Chunks should arrive progressively as the AWS agent generates them, similar to BedrockRuntime's `invoke_model_with_response_stream`:
- Chunk 1 arrives at ~150ms
- Chunk 2 arrives at ~450ms
- Chunk 3 arrives at ~1200ms
- etc.
This enables real-time UX like typing indicators and progressive responses in chatbot applications.
### Current Behavior
All chunks arrive within 1-2ms after waiting for the complete response:
D, [2026-01-21T01:15:34.902128] [AwsBedrockService] Chunk 1 at 25520.21ms, length=31
D, [2026-01-21T01:15:34.902602] [AwsBedrockService] Chunk 3 at 25520.78ms, length=21
D, [2026-01-21T01:15:34.902683] [AwsBedrockService] Chunk 5 at 25520.87ms, length=32
D, [2026-01-21T01:15:34.902734] [AwsBedrockService] Chunk 7 at 25520.92ms, length=56
All 4 chunks delivered within 0.75ms at the 25.5-second mark, despite the agent taking 25+ seconds to generate the response. The SDK is clearly buffering internally.
### Reproduction Steps
```
require 'aws-sdk-bedrockagentcore'
require 'securerandom'
require 'json'
# This reproduces buffering issue with BedrockAgentCore streaming
client = Aws::BedrockAgentCore::Client.new(region: 'us-west-2')
# NOTE: Replace with your actual agent runtime ARN
# Example format: arn:aws:bedrock-agentcore:us-west-2:123456789012:runtime/abc123
agent_arn = ENV.fetch('AGENT_RUNTIME_ARN')
start_time = Time.now
puts "Starting request at #{Time.now}"
resp = client.invoke_agent_runtime(
agent_runtime_arn: agent_arn,
runtime_session_id: SecureRandom.uuid,
payload: JSON.generate({ input: { prompt: 'Tell me a long story' } })
)
puts "\nIterating over response stream:"
chunk_count = 0
resp.response.each do |chunk|
chunk_count += 1
elapsed_ms = ((Time.now - start_time) * 1000).round(2)
puts "Chunk ##{chunk_count} at #{elapsed_ms}ms: #{chunk.bytesize} bytes"
end
puts "\nExpected: Chunks arriving progressively (e.g., 100ms, 500ms, 1000ms)"
puts "Actual: All chunks arriving at approximately the same time (buffering)"
```
Expected: Chunks arriving progressively (100ms, 500ms, 1000ms, etc.)
Actual: All chunks arrive at once after full response completes
### Possible Solution
Add event_stream_handler support similar to aws-sdk-bedrockruntime's invoke_model_with_response_stream:
```
client.invoke_agent_runtime(
agent_runtime_arn: '...',
payload: '...',
event_stream_handler: proc { |stream|
stream.on_event { |event| puts event }
}
)
```
Alternatively, fix resp.response.each to yield chunks as they arrive from the HTTP connection rather than buffering the entire response first.
We've confirmed that raw Net::HTTP with AWS SigV4 signing streams correctly in real-time, proving the service supports it but the SDK buffers.
### Additional Information/Context
This issue makes aws-sdk-bedrockagentcore unusable for real-time applications requiring streaming UX (chatbots, live agents, etc.).
Workaround: We've implemented a monkey patch using raw Net::HTTP that successfully streams in real-time by bypassing the SDK's invoke_agent_runtime method entirely and making direct HTTP requests with streaming enabled.
The BedrockRuntime service gem (aws-sdk-bedrockruntime) has proper event streaming support via event_stream_handler, but BedrockAgentCore lacks this functionality.
### Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version
aws-sdk-bedrockagentcore version 1.14.0
### Environment details (Version of Ruby, OS environment)
- Ruby version: 3.2.6 - Operating System: macOS (Darwin 23.1.0) - Rails: 8.0.2.1
Contributor guide
Assessment
This issue has not been assessed yet.