kelos-dev / kelos-dev/kelos

Workflow: Triage-to-worker handoff loses structured context — add WorkItem metadata enrichment

Open
#455 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

actor/kelos generated-by-kelos kelos/needs-input kind/feature priority/important-longterm triage-accepted
Dominant language
Go
Stars
331
Forks
40
Avg merge
1d 21h
Merged PRs (30d)
70

Description

🤖 Axon Agent @gjkim42

Summary

When the triage agent processes an issue, it generates structured analysis (kind classification, priority assessment, duplicate detection, validity check) and writes it as a GitHub comment. When the worker agent later picks up the same issue, it must re-discover all of this context by re-reading comments, re-analyzing the codebase, and re-classifying the issue from scratch. This is wasteful, error-prone, and creates a fragile coupling where the worker's prompt quality depends on it correctly parsing free-text triage comments.

This proposal identifies the root cause — the WorkItem template model has no mechanism for enriched metadata — and proposes a concrete, incremental solution.

Problem

1. The triage → worker handoff is lossy

The self-development pipeline has a natural multi-agent flow:

axon-triage (discovers issue with needs-actor label)
    → classifies kind, checks validity, finds duplicates
    → posts structured comment, adds axon/needs-input
    → human reviews, adds actor/axon, removes axon/needs-input

axon-workers (discovers issue with actor/axon label)
    → must understand the issue, determine approach, implement fix
    → has NO structured access to triage findings

What triage produces (from self-development/axon-triage.yaml prompt):

  • Kind classification (bug/feature/api/docs)
  • Validity status (STILL VALID / ALREADY FIXED / OUTDATED / DUPLICATE)
  • Evidence with links to relevant PRs, commits, or code
  • Duplicate issue references
  • Recommendation (KEEP OPEN / CLOSE)

What the worker receives: The raw issue body + all comments as a single concatenated string (via {{.Comments}}). The worker must:

  1. Parse the triage comment from other comments
  2. Extract the structured findings from Markdown formatting
  3. Understand which code paths were already investigated
  4. Avoid re-investigating duplicates the triage agent already identified

This is unreliable — the worker agent frequently re-investigates things the triage agent already analyzed, wasting tokens and time.

2. Evidence from actual agent behavior

Looking at recent axon-workers tasks on triaged issues:

  • Workers regularly re-run the same gh pr list --state merged --search queries that triage already performed
  • Workers re-read the same source files to understand the issue context
  • Workers don't consistently leverage the duplicate information from triage
  • The triage comment's "Evidence" section contains exact file paths and line numbers, but workers start fresh exploration anyway

This happens because {{.Comments}} dumps all comments as unstructured text. The agent has no way to know which comment is from triage vs. a human, or which parts are structured findings vs. conversational.

3. The template model can't express enriched metadata

The current WorkItem struct (internal/source/source.go) is fixed:

type WorkItem struct {
    ID       string
    Number   int
    Title    string
    Body     string
    URL      string
    Labels   []string
    Comments string
    Kind     string
    Time     string
    Schedule string
}

And the template data struct in internal/source/prompt.go mirrors this exactly — no extension point exists. There is no way to:

  • Inject computed metadata (e.g., "this issue was triaged, here are the findings")
  • Pass structured data from one spawner's output to another spawner's input
  • Add source-specific enrichment (e.g., Jira custom fields, GitHub PR review status)

Proposed Solution

Part 1: Add Metadata map to WorkItem (API + source change)

Extend the WorkItem struct with an open-ended metadata field:

// internal/source/source.go
type WorkItem struct {
    // ... existing fields ...

    // Metadata contains additional key-value pairs extracted from the source.
    // Available in templates as {{index .Metadata "key"}}.
    Metadata map[string]string
}

And expose it in the template data:

// internal/source/prompt.go — RenderTemplate
data := struct {
    // ... existing fields ...
    Metadata map[string]string
}{
    // ... existing values ...
    Metadata: item.Metadata,
}

This is a backward-compatible, zero-risk change — existing templates don't reference Metadata and won't be affected.

Part 2: GitHub source extracts structured comment metadata

Enhance the GitHub source to parse structured data from specially-formatted comments. When a comment contains an Axon-formatted section (e.g., between <!-- axon-metadata and --> markers), extract it as key-value pairs:

// internal/source/github.go — during comment processing
// Look for structured metadata in comments matching the agent comment format
if strings.Contains(comment.Body, "## Axon Triage Report") {
    meta := extractTriageMetadata(comment.Body)
    // meta might contain:
    //   "triage-kind": "kind/bug"
    //   "triage-status": "STILL VALID"
    //   "triage-recommendation": "KEEP OPEN"
    //   "triage-duplicates": "#287, #298"
    for k, v := range meta {
        item.Metadata[k] = v
    }
}

Alternative (cleaner): Have the triage agent embed structured metadata in an HTML comment:

<!-- axon-metadata
triage-kind: kind/bug
triage-status: STILL VALID
triage-duplicates: #287
triage-evidence: The spawner dedup logic at cmd/axon-spawner/main.go:170 still checks all tasks
-->

## Axon Triage Report
**Kind**: kind/bug
...

The GitHub source would parse these HTML comment blocks into WorkItem.Metadata. This is invisible to human readers but machine-readable.

Part 3: Worker prompt leverages triage context

Update self-development/axon-workers.yaml to use triage metadata when available:

promptTemplate: |
  You are a coding agent. ...

  {{- if index .Metadata "triage-kind"}}

  ## Triage Context (from automated triage)
  - Classification: {{index .Metadata "triage-kind"}}
  - Status: {{index .Metadata "triage-status"}}
  - Recommendation: {{index .Metadata "triage-recommendation"}}
  {{- if index .Metadata "triage-duplicates"}}
  - Related issues: {{index .Metadata "triage-duplicates"}}
  {{- end}}
  {{- if index .Metadata "triage-evidence"}}
  - Key findings: {{index .Metadata "triage-evidence"}}
  {{- end}}

  Use this triage context to skip redundant investigation. The triage agent
  has already verified this issue is valid and classified it.
  {{- end}}

  Task:
  ...
Part 4: Triage prompt emits structured metadata

Update self-development/axon-triage.yaml to include a metadata block in its comment:

promptTemplate: |
  ...
  Format the comment as:

Axon Triage Report

...

Why This Matters Beyond Self-Development

The Metadata extension is useful for any Axon user, not just self-development:

Use case 1: Jira custom fields

Jira issues have rich custom fields (story points, sprint, components, epic link) that aren't exposed in the current WorkItem. With Metadata:

promptTemplate: |
  Fix Jira ticket {{.Title}}
  Priority: {{index .Metadata "priority"}}
  Component: {{index .Metadata "component"}}
  Story points: {{index .Metadata "story-points"}}
Use case 2: GitHub PR review context

When a spawner watches PRs, the review status and requested changes are valuable:

promptTemplate: |
  PR #{{.Number}} has requested changes.
  Review summary: {{index .Metadata "review-body"}}
  Reviewer: {{index .Metadata "reviewer"}}
Use case 3: Issue template parsing

Many repos use structured issue templates with YAML frontmatter or checkbox sections. Metadata extraction could parse these:

promptTemplate: |
  Bug report for {{.Title}}
  Severity: {{index .Metadata "severity"}}
  Steps to reproduce: {{index .Metadata "steps-to-reproduce"}}
  Expected behavior: {{index .Metadata "expected-behavior"}}

Implementation Scope

Change Files Estimated Lines
Add Metadata to WorkItem internal/source/source.go +3
Expose in template internal/source/prompt.go +2
GitHub source: init Metadata map internal/source/github.go +5
GitHub source: extract comment metadata internal/source/github.go +40
Jira source: extract custom fields internal/source/jira.go +20
Triage prompt: emit metadata self-development/axon-triage.yaml +10
Worker prompt: consume metadata self-development/axon-workers.yaml +15
Tests internal/source/*_test.go +50
Total ~145 lines

Backward Compatibility

  • Metadata field is optional (nil map by default)
  • Existing templates that don't reference {{.Metadata}} are unaffected
  • Existing GitHub comments without <!-- axon-metadata --> blocks produce no metadata
  • No CRD changes needed — this is an internal source/template enhancement
  • No API version bump required

Alternatives Considered

Alternative: Use Workspace.Files to inject context

Workspace files field can write arbitrary files into the repo before the agent starts. A pre-processing step could write triage findings to a file. But this requires a separate mechanism to run the pre-processing, and doesn't compose with the template system.

Alternative: Use AgentConfig.agentsMD for context

AgentConfig is shared across all tasks from a spawner — it can't carry per-issue context. You'd need a separate AgentConfig per issue, which defeats the purpose.

Alternative: Let the worker parse comments itself

This is the status quo. It works sometimes but is unreliable because:

  • Comment parsing is a non-trivial NLP task for the agent
  • The agent wastes tokens reading and interpreting comments instead of coding
  • Different comment formats from different agents (triage, human, bot) are ambiguous

Related Issues

Issue Relationship
#437 (Triage lifecycle) Complementary — #437 extends triage's label management; this proposal extends triage's data output
#427 (Multi-step pipelines) Complementary — pipelines pass data between tasks via dependsOn; this passes data between spawners via work item metadata
#283 (taskCompletion trigger) Complementary — taskCompletion chains spawners; metadata enrichment ensures context flows through the chain
#400 (Multi-run) Addresses the same root cause — agents forget steps because they lack context. Metadata enrichment reduces what agents need to discover

References

  • WorkItem struct: internal/source/source.go:5-16
  • Template rendering: internal/source/prompt.go:29-73
  • GitHub comment extraction: internal/source/github.go:86-116
  • Triage prompt output format: self-development/axon-triage.yaml:78-96
  • Worker prompt context needs: self-development/axon-workers.yaml:41-82
  • Jira custom fields: internal/source/jira.go:82-107

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with WorkItem and template rendering in internal/source/source.go and internal/source/prompt.go, then inspect comment extraction in internal/source/github.go and the referenced triage and worker YAML prompts. Decide the structured metadata format, add coverage in internal/source/*_test.go, and verify that triage metadata is extracted and available to worker templates without affecting existing templates.

Written by the indexing model from the issue text.

Assessment

Tech stack
github, go
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.