awslabs / awslabs/cli-agent-orchestrator

[Workflow] Generate Python workflows using discovered agent profiles

Open
#601 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.3k
Forks
267
Avg merge
1d 23h
Merged PRs (30d)
70

Description

## Overview

Let a user describe a process to an authoring agent and receive a validated Python workflow that uses suitable existing CAO agent profiles.

The authoring agent should discover profiles with `find_profiles` / `cao profile find`, select workers for each step, generate the workflow, and present the resolved plan for approval.

Parent roadmap: #583
Related dynamic-supervisor proposal: #12

## Problem

CAO can already generate and execute Python workflows, and it can search installed profiles by capability. These features are not yet connected into one authoring experience.

Today, workflow authors must already know profile names or use generic profiles such as `developer` and `reviewer`. This creates several problems:

- users must understand CAO's installed profile catalog;
- generated workflows may hardcode unsuitable or nonexistent profiles;
- adding a specialized profile still requires manually updating the workflow; and
- profile selection is not explained or frozen before execution.

## Proposed Direction

When a user asks an agent to build a workflow:

1. Interpret the requested outcome and identify the capabilities needed by each step.
2. Search existing profiles using `find_profiles`.
3. Select and validate a profile for each step.
4. Generate a deterministic Python workflow referencing those profiles.
5. Validate the generated artifact.
6. Present the steps, selected profiles, permissions, limits, and selection rationale.
7. Run only after explicit user approval.

Profile discovery happens during authoring, not during workflow execution. Once approved, the workflow and execution plan retain the selected profile identities so resume and rerun do not silently select different workers.

Profile metadata returned by discovery is untrusted data and must not be treated as instructions.

## No-Match Behavior

If no suitable profile is found, the authoring agent must explain the capability gap and either:

- propose an existing generic profile with its limitations; or
- ask the user to install or create an appropriate profile.

The first version must not generate an ephemeral profile automatically.

## Acceptance Criteria

- [ ] A user can request a workflow without naming profiles or choosing Python versus YAML.
- [ ] The authoring agent searches installed profiles for each required capability.
- [ ] Every selected profile exists and can be loaded before the workflow is presented.
- [ ] The generated artifact is a Python workflow and passes `cao workflow validate`.
- [ ] The approval view identifies every step's profile, provider, permissions, and selection rationale.
- [ ] The approved workflow does not repeat profile discovery during execution or resume.
- [ ] A missing profile produces an actionable explanation rather than an invalid workflow.
- [ ] At least one automated scenario generates and validates a multi-profile workflow against a controlled profile catalog.

## Alternatives Considered

### Runtime profile selection

A workflow could search for a worker every time it runs. This was rejected for the first version because the same workflow could resolve to different workers, permissions, or behavior after profile changes.

### Ephemeral profile generation

The agent could create a worker with a generated prompt, tools, and skills. This remains relevant to #12, but it requires separate permission, review, installation, and cleanup semantics.

### Require users to select profiles

This preserves explicit control but leaves the current authoring friction unresolved.

## Risks and Open Questions

The largest risk is incorrect selection caused by incomplete or stale profile metadata. The initial validation should use a controlled catalog and inspect whether the generated plan selects appropriate workers.

Open questions:

- What minimum confidence or evidence should be required before selecting a profile?
- Should the approved plan freeze only the profile name or also the profile content hash?
- How should an unavailable profile be handled when a saved workflow is run later?
- Should profile selection rationale be retained with the artifact or only in the execution manifest?

## Out of Scope

- Creating ephemeral worker profiles.
- Extending `cao profile create`.
- Enumerating currently running agent instances.
- Runtime profile reselection.
- Autonomous execution without approval.
- YAML workflow generation.
- Changes to profile-search ranking.

Contributor guide

Open the contributing guide

Research direction

Start with the `find_profiles`/`cao profile find` entry points and the existing Python workflow-generation path, then use a controlled profile catalog to exercise profile selection and no-match behavior. Done means the generated artifact passes `cao workflow validate`, presents the resolved plan for approval, and retains selected profiles without rediscovery during execution or resume.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.