(bedrock-agentcore): L2 constructs for Evaluator and OnlineEvaluationConfig
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the feature
The @aws-cdk/aws-bedrock-agentcore-alpha package has L2 constructs for Runtime, Browser, Code Interpreter, Gateway, and Memory — but not for Evaluations. The L1 resources AWS::BedrockAgentCore::Evaluator and AWS::BedrockAgentCore::OnlineEvaluationConfig exist but have no L2 wrappers.
Requesting L2 constructs for:
Evaluator — custom evaluators (LLM-as-a-Judge and code-based)
OnlineEvaluationConfig — continuous monitoring of agent performance
### Use Case
I'm building an AI agent on AgentCore and need to set up continuous quality monitoring using CDK. Currently I have to use the L1 CfnEvaluator and CfnOnlineEvaluationConfig directly, manually constructing IAM roles with the correct permissions, wiring up CloudWatch Logs data sources, and handling the mapping between evaluator references and their IDs. An L2 construct would provide sensible defaults (auto-created execution role with scoped permissions, built-in evaluator references, inference profile detection for model ARNs) and a developer-friendly API consistent with the existing Runtime, Browser, Gateway, and Memory constructs in the package.
### Proposed Solution
_No response_
### Other Information
_No response_
### Acknowledgements
- [x] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
### AWS CDK Library version (aws-cdk-lib)
2.250.0
### AWS CDK CLI version
2.1118.4
### Environment details (OS name and version, etc.)
Amazon Linux 2023 (EC2), Node.js 20.20.2, TypeScript 5.5.4, CDK CLI 2.1118.4
Contributor guide
Research direction
Start by inspecting the existing Runtime, Browser, Code Interpreter, Gateway, and Memory L2 constructs in @aws-cdk/aws-bedrock-agentcore-alpha, then compare their APIs with the AWS::BedrockAgentCore::Evaluator and AWS::BedrockAgentCore::OnlineEvaluationConfig L1 resources. Done means both resources have consistent developer-facing L2 wrappers with appropriate IAM, logging, evaluator-reference, and model-ARN behavior, covered by the package's construct tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100