aws / aws/bedrock-agentcore-starter-toolkit

[BUG] Infinite loop of Cognito M2M and Bedrock requests after enabling Observability

Open
#420 2 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
Python
Stars
508
Forks
155
Avg merge
8h 50m
Merged PRs (30d)
4

Description

**Describe the bug**
After creating a production-ready agent using the starter toolkit (Terraform + Strands Agents SDK) and configuring observability following the official documentation, I observed a critical issue.

Approximately one day after deployment and invoking the runtime via the Agent Sandbox (invoke ~35 times), CloudTrail logs showed a massive volume of Cognito M2M requests (~2000 Requests) and Bedrock requests. This appears to be an infinite loop, resulting in significant unexpected billing costs.

**To Reproduce**
It might not be able to be reproduced, but this is what I did:

1. Initialize a new project using `agentcore create` with the following selections:
```text
How would you like to start?
> A production-ready agent defined with Terraform or CDK

What agent framework should we use?
> Strands Agents SDK

Which model provider will power your agent?
> Amazon Bedrock

Which IaC provider will define your AgentCore resources?
> Terraform

Initialize a new git repository?
> Yes
```

2. Configure observability by following the guide: [Amazon Bedrock AgentCore Observability Configuration](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/observability-configure.html).
3. Deploy the resources using Terraform.
4. Invoke the agent runtime using the Agent Sandbox.
5. Wait for a period of time (approx. 24 hours).
6. Observe CloudTrail for a spike in `Cognito` (M2M token requests) and `Bedrock` API calls.

**Expected behavior**
The agent should process the invocation and remain idle or log telemetry data without triggering further invocations. Enabling observability should not cause a recursive loop of self-invocation or continuous authentication requests.

**Impact**
- **High Severity:** This bug causes an infinite loop of API calls.
- **Financial Impact:** Generated a large unexpected bill due to continuous Cognito and Bedrock usage.

**Error Output**
There is no explicit error output in the console. The issue is observed via CloudTrail logs showing repetitive:
- Cognito M2M token requests (token_POST)
- Cost Explorer :(

**Environment:**
- **Python version:** 3.10.0
- **Package versions:**
- `bedrock-agentcore`: 1.1.2
- `strands-agents`: 1.21.0

Contributor guide

Open the contributing guide

Research direction

Start with the agentcore create production-ready Terraform template, then compare its observability configuration with the Amazon Bedrock AgentCore Observability Configuration guide. Reproduce the deployment and Agent Sandbox invocation if possible, and inspect CloudTrail for repeated Cognito token_POST and Bedrock calls over time. Done means identifying the source of the repeated requests and confirming that an observability-enabled agent remains idle after an invocation.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, terraform
Domain
backend, cloud, observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.