kelos-dev / kelos-dev/kelos

Prompt Tuning: kelos-triage and kelos-squash-commits missing /kelos needs-input comment for worker exclusion

Open
#634 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

actor/kelos generated-by-kelos kelos/needs-input kind/bug priority/important-longterm triage-accepted
Dominant language
Go
Stars
331
Forks
40
Avg merge
1d 21h
Merged PRs (30d)
70

Description

🤖 Kelos Self-Update Agent @gjkim42

Area: Prompt Tuning

Summary

Two TaskSpawner prompts (kelos-triage and kelos-squash-commits) fail to instruct their agents to post the /kelos needs-input comment that the kelos-workers spawner relies on to prevent premature pick-up. This creates a race condition where worker agents can start on issues before a maintainer has reviewed triage results or before a squash operation's follow-up review is complete.

Findings

1. kelos-triage.yaml prompt missing /kelos needs-input comment and kelos/needs-input label

README expectation (self-development/README.md, line 132):

Posts a single triage comment with its findings, adds the kelos/needs-input label (to prevent re-triage), and posts a /kelos needs-input comment (to prevent workers from picking up the issue before maintainer review).

Actual prompt (kelos-triage.yaml, lines 142-150): The prompt only instructs the agent to add triage-accepted and optionally actor/kelos, then remove needs-triage and needs-actor:

If recommending `actor/kelos`:
  `gh issue edit {{.Number}} --add-label triage-accepted --add-label actor/kelos --remove-label needs-triage --remove-label needs-actor`

If leaving `needs-actor`:
  `gh issue edit {{.Number}} --add-label triage-accepted --remove-label needs-triage`

What's missing from the prompt:

  • No instruction to add the kelos/needs-input label
  • No instruction to post a /kelos needs-input comment

Impact: When triage recommends actor/kelos, it adds the actor/kelos label (which is the kelos-workers trigger label) but does NOT post /kelos needs-input. The kelos-workers spawner uses excludeComments: ["/kelos needs-input"] to defer issues that need human review. Without this comment, the workers spawner could immediately pick up a triaged issue before the maintainer has reviewed the triage result or posted /kelos pick-up.

In practice, the triggerComment: "/kelos pick-up" requirement on kelos-workers currently prevents this — the worker won't start until a maintainer comments /kelos pick-up. However, this is a defense-in-depth gap: if triggerComment were ever removed or loosened, the missing /kelos needs-input comment would become a live race condition.

More concretely, the README documents this as existing behavior, which means the intention was always to have this gate. The prompt should match the documented design.

Proposed fix: Add to the triage prompt's label-update section (after the gh issue edit commands):

After updating labels, post a comment to signal that triage is complete and the issue needs maintainer review:
  `gh issue comment {{.Number}} --body "/kelos needs-input\n\nTriage complete. Please review the triage report above and `/kelos pick-up` if you want a worker agent to implement this."`

And when recommending actor/kelos, include kelos/needs-input in the label update:

  `gh issue edit {{.Number}} --add-label triage-accepted --add-label actor/kelos --add-label kelos/needs-input --remove-label needs-triage --remove-label needs-actor`
2. kelos-squash-commits.yaml uses label instead of comment for needs-input signaling

Current prompt (kelos-squash-commits.yaml, lines 87-89):

7. Add `kelos/needs-input` label to the linked issue:
   - Extract the linked issue number from the PR body (look for `Closes #N`, `Fixes #N`, etc.)
   - Run: `gh issue edit <issue-number> --add-label kelos/needs-input`

Problem: The kelos-workers spawner filters on excludeComments: ["/kelos needs-input"], not excludeLabels: ["kelos/needs-input"]. Adding the kelos/needs-input label does not trigger the workers' comment-based exclusion filter.

The label and the comment serve different roles in the system:

  • Label (kelos/needs-input): visual indicator on the GitHub UI that the issue needs human attention
  • Comment (/kelos needs-input): programmatic signal that the kelos-workers spawner reads via excludeComments to prevent premature pick-up

The squash agent should do both — add the label AND post the comment — to fully signal that the issue needs maintainer review after the squash.

Evidence of the dual-signal pattern: The kelos-workers.yaml prompt itself (line 119) instructs the worker to:

Post a `/kelos needs-input` comment on the issue (gh issue comment {{.Number}} --body "/kelos needs-input")

This posts a comment, not a label. The squash agent should follow the same pattern.

Proposed fix: Replace step 7 with:

7. Signal that the linked issue needs maintainer review:
   - Extract the linked issue number from the PR body (look for `Closes #N`, `Fixes #N`, etc.)
   - Add the label: `gh issue edit <issue-number> --add-label kelos/needs-input`
   - Post the exclusion comment: `gh issue comment <issue-number> --body "/kelos needs-input\n\nCommits squashed. Please review the PR."`

Summary of changes needed

File Issue Fix
kelos-triage.yaml No /kelos needs-input comment or kelos/needs-input label after triage Add comment and label instructions to both actor/kelos and needs-actor code paths
kelos-squash-commits.yaml Step 7 adds label but not comment Add /kelos needs-input comment alongside the label

Relationship to existing issues/PRs

  • Issue #455 (triage-to-worker metadata enrichment) — different concern; #455 is about passing structured triage data to workers, not about the /kelos needs-input exclusion gate
  • Issue #592 (workers step 7a template variable misuse) — different concern; covers {{.Number}} in gh pr edit
  • PR #551 (task template sync) — covers task template drift, not the spawner prompts themselves
  • Issue #601 (AgentConfig co-location and drift) — covers agentsMD drift, not prompt logic

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with self-development/README.md and compare the referenced sections of kelos-triage.yaml and kelos-squash-commits.yaml. Verify how kelos-workers.yaml uses the /kelos needs-input comment, then update both prompts so the documented label and comment signals are emitted. Done means both prompt paths provide the label and exclusion comment for maintainer review.

Written by the indexing model from the issue text.

Assessment

Tech stack
github
Domain
devtools
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.