awslabs / awslabs/cli-agent-orchestrator

[Feat] Add an experimental AgentCore elastic worker backend

Open
#694 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.3k
Forks
267
Avg merge
1d 23h
Merged PRs (30d)
70

Description

## Overview

CAO's EKS elastic-worker branch demonstrates remote, isolated worker execution through Kubernetes Jobs. Amazon Bedrock AgentCore Runtime now provides interactive command shells, persistent filesystems, isolated sessions, streaming, and managed lifecycle controls that could offer a lower-operations hosting option.

Add an experimental AgentCore backend that allows a CAO supervisor to dispatch work to an isolated AgentCore Runtime session.

## Problem

Running elastic CAO workers currently requires operating Kubernetes infrastructure, including a broker, Jobs, storage, networking, RBAC, and cleanup logic.

Users who already operate on AWS need a managed execution option that preserves CAO's provider-neutral orchestration model without requiring them to manage an EKS cluster.

## User Stories

- As a CAO operator, I want to run elastic workers on AgentCore so that I do not need to manage worker compute directly.
- As a CAO user, I want remote workers to stream progress and return authoritative outcomes like local workers.
- As a security administrator, I want agents restricted to approved runtime, identity, network, and storage configurations.

## Proposed Direction

Introduce AgentCore as an experimental elastic worker backend, informed by the EKS branch's remote-target, lease, and worker lifecycle concepts.

For the first slice:

- Keep the CAO supervisor outside AgentCore.
- Dispatch one worker into an approved AgentCore Runtime.
- Run the provider CLI through an AgentCore interactive command shell.
- Use managed session storage, EFS, or S3 Files for the workspace.
- Stream worker output and status to the supervisor.
- Return a verifiable completion callback.
- Stop or clean up the Runtime session after completion.

The supervisor must select from operator-approved Runtime configurations. Agent-controlled input must not directly select arbitrary images, IAM roles, networks, or storage.

## Acceptance Criteria

- [ ] A supervisor can dispatch an assignment to a configured AgentCore backend.
- [ ] The assignment has stable CAO and AgentCore session identifiers.
- [ ] A supported provider CLI can complete a real repository task in the Runtime.
- [ ] Worker progress and terminal output are available while the task runs.
- [ ] The worker returns an explicit success or failure outcome with supporting execution evidence.
- [ ] Temporary disconnection and shell reconnection are handled.
- [ ] Workspace files survive for the required assignment lifecycle.
- [ ] Timeout, cancellation, retryable conflict, and cleanup behavior are defined.
- [ ] Runtime, IAM role, networking, and storage choices are constrained by operator configuration.
- [ ] Documentation describes prerequisites, lifecycle limits, security boundaries, and expected costs.

## Validation

Prototype one fixed Runtime image and complete one end-to-end remote assignment. Measure:

- Worker startup latency
- Streaming and callback reliability
- Shell reconnection behavior
- Workspace persistence
- Cancellation and cleanup reliability
- Runtime and storage cost per assignment

## Out of Scope

- Hosting the complete CAO supervisor on AgentCore
- Dynamically creating Runtime images or IAM roles
- Supporting every CAO provider initially
- Replacing CAO memory with AgentCore Memory
- AgentCore Harness integration
- Solving CAO's terminal false-success signal solely through hosting changes

## Risks and Open Questions

- How should AgentCore workers reach the CAO supervisor without a Kubernetes Service equivalent?
- Can Runtime shell events provide a stronger completion signal than the current TUI heuristic?
- Should longer sessions use microVMs or AgentCore Instances?
- Which workspace option best preserves CAO worktree isolation?
- How should OAuth and device-flow provider credentials be supplied?
- What lease or reconciliation mechanism detects abandoned Runtime sessions?

## References

- [AgentCore Runtime](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/agents-tools-runtime.html)
- [Interactive shells](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/runtime-get-started-command-shell.html)
- [Filesystem configurations](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/runtime-filesystem-configurations.html)
- [Runtime Instances](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/runtime-instances-how-it-works.html)
- [EKS elastic-worker precedent](https://github.com/awslabs/cli-agent-orchestrator/tree/joeguo/eks-elastic-contrib)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the EKS elastic-worker precedent and the linked AgentCore Runtime, interactive shell, filesystem, and Runtime Instances documentation. Trace the existing supervisor, remote-target, lease, and worker-lifecycle concepts before defining the backend boundary. Done means one fixed Runtime can execute a real repository assignment with streaming, reconnection, persistence, explicit outcome evidence, constrained configuration, and cleanup.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
backend, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.