awslabs / awslabs/cli-agent-orchestrator
[Workflow] Generate Python workflows using discovered agent profiles
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 267
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 70
Description
## Overview
Let a user describe a process to an authoring agent and receive a validated Python workflow that uses suitable existing CAO agent profiles.
The authoring agent should discover profiles with `find_profiles` / `cao profile find`, select workers for each step, generate the workflow, and present the resolved plan for approval.
Parent roadmap: #583
Related dynamic-supervisor proposal: #12
## Problem
CAO can already generate and execute Python workflows, and it can search installed profiles by capability. These features are not yet connected into one authoring experience.
Today, workflow authors must already know profile names or use generic profiles such as `developer` and `reviewer`. This creates several problems:
- users must understand CAO's installed profile catalog;
- generated workflows may hardcode unsuitable or nonexistent profiles;
- adding a specialized profile still requires manually updating the workflow; and
- profile selection is not explained or frozen before execution.
## Proposed Direction
When a user asks an agent to build a workflow:
1. Interpret the requested outcome and identify the capabilities needed by each step.
2. Search existing profiles using `find_profiles`.
3. Select and validate a profile for each step.
4. Generate a deterministic Python workflow referencing those profiles.
5. Validate the generated artifact.
6. Present the steps, selected profiles, permissions, limits, and selection rationale.
7. Run only after explicit user approval.
Profile discovery happens during authoring, not during workflow execution. Once approved, the workflow and execution plan retain the selected profile identities so resume and rerun do not silently select different workers.
Profile metadata returned by discovery is untrusted data and must not be treated as instructions.
## No-Match Behavior
If no suitable profile is found, the authoring agent must explain the capability gap and either:
- propose an existing generic profile with its limitations; or
- ask the user to install or create an appropriate profile.
The first version must not generate an ephemeral profile automatically.
## Acceptance Criteria
- [ ] A user can request a workflow without naming profiles or choosing Python versus YAML.
- [ ] The authoring agent searches installed profiles for each required capability.
- [ ] Every selected profile exists and can be loaded before the workflow is presented.
- [ ] The generated artifact is a Python workflow and passes `cao workflow validate`.
- [ ] The approval view identifies every step's profile, provider, permissions, and selection rationale.
- [ ] The approved workflow does not repeat profile discovery during execution or resume.
- [ ] A missing profile produces an actionable explanation rather than an invalid workflow.
- [ ] At least one automated scenario generates and validates a multi-profile workflow against a controlled profile catalog.
## Alternatives Considered
### Runtime profile selection
A workflow could search for a worker every time it runs. This was rejected for the first version because the same workflow could resolve to different workers, permissions, or behavior after profile changes.
### Ephemeral profile generation
The agent could create a worker with a generated prompt, tools, and skills. This remains relevant to #12, but it requires separate permission, review, installation, and cleanup semantics.
### Require users to select profiles
This preserves explicit control but leaves the current authoring friction unresolved.
## Risks and Open Questions
The largest risk is incorrect selection caused by incomplete or stale profile metadata. The initial validation should use a controlled catalog and inspect whether the generated plan selects appropriate workers.
Open questions:
- What minimum confidence or evidence should be required before selecting a profile?
- Should the approved plan freeze only the profile name or also the profile content hash?
- How should an unavailable profile be handled when a saved workflow is run later?
- Should profile selection rationale be retained with the artifact or only in the execution manifest?
## Out of Scope
- Creating ephemeral worker profiles.
- Extending `cao profile create`.
- Enumerating currently running agent instances.
- Runtime profile reselection.
- Autonomous execution without approval.
- YAML workflow generation.
- Changes to profile-search ranking.
Contributor guide
Research direction
Start with the `find_profiles`/`cao profile find` entry points and the existing Python workflow-generation path, then use a controlled profile catalog to exercise profile selection and no-match behavior. Done means the generated artifact passes `cao workflow validate`, presents the resolved plan for approval, and retains selected profiles without rediscovery during execution or resume.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100