awslabs / awslabs/agentcore-samples
Amazon Bedrock AgentCore Samples - AgentCore pre-emptive vm warmup
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 1.3k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 30
Description
**Describe the tutorial request**
A clear and concise description of what is being requested.
```
A demonstration application showcasing how pre-emptive warmup of AgentCore runtime reduces latency in agentic chatbots.
AgentCore Runtime can deploy agent code using code deployment or container deployment.
When AgentCore Runtime receives the first invocation, it allocates a new VM. In case of container deployment there are 10 VM in a warm pool, and up to 10 new sessions are served from that pool, with sub-second cold starts. From the 11th session, new VMs must be allocated, the container downloaded and started. This will make the cold start longer and dependent on the size of the container.
For code deployment, AgentCore Runtime manages a larger pool of instances and only pushes the agent code. This ensure a more consistent startup time and higher session per second limit, but the cold start will be around 2-3 seconds.
For agentic chat applications, a low TTFT is paramount for a good user experience. This sample show a pattern to pre-emptively warmup agent core instances so that when the user actually send the message to the agent, the VM has already been initialized.
```
**Is this feature already supported by the AgentCore service?**
- [ ] Yes
- [x] No
If `yes`, please provide any appropriate documentation link.
Contributor guide
Assessment
This issue has not been assessed yet.