kelos-dev / kelos-dev/kelos

Prompt Tuning: kelos-self-update has contradictory 'Only modify files under...' constraint despite being issue-only

Open Beginner friendly
#1,122 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

actor/kelos generated-by-kelos kind/cleanup priority/important-longterm triage-accepted
Dominant language
Go
Stars
331
Forks
40
Avg merge
1d 21h
Merged PRs (30d)
70

Description

🤖 Kelos Self-Update Agent @gjkim42

Problem

self-development/kelos-self-update.yaml contains contradictory wording in its promptTemplate. The Actions section says the agent must only create issues, never PRs, while the Constraints section then states the agent should "Only modify files under self-development/…". Since the agent has no path to commit/push (it only creates issues), the modify-files line is dead text at best and actively misleading at worst — it suggests the agent should be making file edits that the ephemeral container will discard.

Evidence

self-development/kelos-self-update.yaml lines 97–116:

```
Actions:

  • If you find improvements, create a GitHub issue with your findings and proposals:
    ```
    gh issue create --title "<title>" --body "" --label generated-by-kelos
    ```
  • Do NOT create PRs. Only create issues

Constraints:

  • Focus on ONE area per run, do it thoroughly
  • Only modify files under `self-development/` and related workflow files under `.github/workflows/`
  • Do not create duplicate issues — check existing issues first with `gh issue list`

    ```

Two further pieces of evidence that the agent really is issue-only:

  1. The TaskSpawner has no branch: field, no GIT_AUTHOR_* env vars, and no reporting: block (compare with kelos-image-update.yaml and kelos-config-update.yaml, which both create PRs and have those settings).
  2. Closed issue #472 documents the explicit transition of kelos-self-update from PR-creating to issue-only ("README stale after kelos-self-update changed to issue-only").

Recent kelos-self-update output also confirms practice: #1116, #1109, #1089, #1019, #968 are all issues, no PRs. So the "modify files" constraint has been silently ignored — which makes it pure noise that competes for the model's attention against the real instructions.

For comparison, the sibling cron-only "create issues only" agents do not carry an analogous constraint:

  • kelos-fake-strategist.yaml Constraints section has no "Only modify files" line
  • kelos-fake-user.yaml Constraints section has no "Only modify files" line

So kelos-self-update is the only issue-only agent whose prompt implies file modification is part of the workflow.

Why this matters

  • A model reading "Only modify files under `self-development/`" together with "Do NOT create PRs" must reconcile two contradictory directives. Some runs may waste budget editing files that vanish; others may be confused into proposing scope outside what the constraint really intends (which appears to be: the proposals in the issue should be scoped to self-development/ and .github/workflows/, not that the agent should literally modify files).
  • The 4 focus areas (Prompt Tuning / Configuration Alignment / Workflow Completeness / Task Template Maintenance) all already implicitly target self-development/ and .github/workflows/. The scope constraint, if kept at all, should be reworded to reflect "scope of proposed changes" rather than "files you are allowed to modify".

Proposal

Replace the misleading constraint with a scope-of-proposals constraint. Concretely, change self-development/kelos-self-update.yaml lines 105–106:

Before:
```
Constraints:

  • Focus on ONE area per run, do it thoroughly
  • Only modify files under `self-development/` and related workflow files under `.github/workflows/`
    ```

After (option A — drop the line entirely, matching the other issue-only agents):
```
Constraints:

  • Focus on ONE area per run, do it thoroughly
    ```

After (option B — rewrite as a scope-of-proposals constraint):
```
Constraints:

  • Focus on ONE area per run, do it thoroughly
  • Scope every proposed change to files under `self-development/` or workflow files under `.github/workflows/` — do not propose changes outside these paths
    ```

Option B is more faithful to apparent intent and keeps the scoping signal. Either option removes the contradiction with "Do NOT create PRs. Only create issues."

Acceptance criteria

  • self-development/kelos-self-update.yaml no longer contains text that instructs the agent to modify files while simultaneously forbidding it from creating PRs
  • If a scope constraint is retained, it explicitly describes the scope of proposed changes, not the scope of file modifications
  • No change to the four focus areas (Prompt Tuning, Configuration Alignment, Workflow Completeness, Task Template Maintenance)
  • No drift introduced in self-development/README.md (the README does not currently describe this constraint, so no update needed)

/kind cleanup

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Open self-development/kelos-self-update.yaml around lines 97–116 and compare the issue-only constraints with kelos-fake-strategist.yaml and kelos-fake-user.yaml. Choose whether to remove the file-modification line or rewrite it as a proposed-change scope, then verify the four focus areas and self-development/README.md remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
ci-cd, devops
Issue type
Refactor
Difficulty
1/5
Estimated time
Under an hour
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.