Prompt Tuning: kelos-self-update has contradictory 'Only modify files under...' constraint despite being issue-only
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 331
- Forks
- 40
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 70
Description
🤖 Kelos Self-Update Agent @gjkim42
Problem
self-development/kelos-self-update.yaml contains contradictory wording in its promptTemplate. The Actions section says the agent must only create issues, never PRs, while the Constraints section then states the agent should "Only modify files under self-development/…". Since the agent has no path to commit/push (it only creates issues), the modify-files line is dead text at best and actively misleading at worst — it suggests the agent should be making file edits that the ephemeral container will discard.
Evidence
self-development/kelos-self-update.yaml lines 97–116:
```
Actions:
- If you find improvements, create a GitHub issue with your findings and proposals:
```
gh issue create --title "<title>" --body "" --label generated-by-kelos
``` - Do NOT create PRs. Only create issues
Constraints:
- Focus on ONE area per run, do it thoroughly
- Only modify files under `self-development/` and related workflow files under `.github/workflows/`
- Do not create duplicate issues — check existing issues first with `gh issue list`
…
```
Two further pieces of evidence that the agent really is issue-only:
- The TaskSpawner has no
branch:field, noGIT_AUTHOR_*env vars, and noreporting:block (compare withkelos-image-update.yamlandkelos-config-update.yaml, which both create PRs and have those settings). - Closed issue #472 documents the explicit transition of kelos-self-update from PR-creating to issue-only ("README stale after kelos-self-update changed to issue-only").
Recent kelos-self-update output also confirms practice: #1116, #1109, #1089, #1019, #968 are all issues, no PRs. So the "modify files" constraint has been silently ignored — which makes it pure noise that competes for the model's attention against the real instructions.
For comparison, the sibling cron-only "create issues only" agents do not carry an analogous constraint:
kelos-fake-strategist.yamlConstraints section has no "Only modify files" linekelos-fake-user.yamlConstraints section has no "Only modify files" line
So kelos-self-update is the only issue-only agent whose prompt implies file modification is part of the workflow.
Why this matters
- A model reading "Only modify files under `self-development/`" together with "Do NOT create PRs" must reconcile two contradictory directives. Some runs may waste budget editing files that vanish; others may be confused into proposing scope outside what the constraint really intends (which appears to be: the proposals in the issue should be scoped to
self-development/and.github/workflows/, not that the agent should literally modify files). - The 4 focus areas (Prompt Tuning / Configuration Alignment / Workflow Completeness / Task Template Maintenance) all already implicitly target
self-development/and.github/workflows/. The scope constraint, if kept at all, should be reworded to reflect "scope of proposed changes" rather than "files you are allowed to modify".
Proposal
Replace the misleading constraint with a scope-of-proposals constraint. Concretely, change self-development/kelos-self-update.yaml lines 105–106:
Before:
```
Constraints:
- Focus on ONE area per run, do it thoroughly
- Only modify files under `self-development/` and related workflow files under `.github/workflows/`
```
After (option A — drop the line entirely, matching the other issue-only agents):
```
Constraints:
- Focus on ONE area per run, do it thoroughly
```
After (option B — rewrite as a scope-of-proposals constraint):
```
Constraints:
- Focus on ONE area per run, do it thoroughly
- Scope every proposed change to files under `self-development/` or workflow files under `.github/workflows/` — do not propose changes outside these paths
```
Option B is more faithful to apparent intent and keeps the scoping signal. Either option removes the contradiction with "Do NOT create PRs. Only create issues."
Acceptance criteria
-
self-development/kelos-self-update.yamlno longer contains text that instructs the agent to modify files while simultaneously forbidding it from creating PRs - If a scope constraint is retained, it explicitly describes the scope of proposed changes, not the scope of file modifications
- No change to the four focus areas (Prompt Tuning, Configuration Alignment, Workflow Completeness, Task Template Maintenance)
- No drift introduced in
self-development/README.md(the README does not currently describe this constraint, so no update needed)
/kind cleanup
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Open self-development/kelos-self-update.yaml around lines 97–116 and compare the issue-only constraints with kelos-fake-strategist.yaml and kelos-fake-user.yaml. Choose whether to remove the file-modification line or rewrite it as a proposed-change scope, then verify the four focus areas and self-development/README.md remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions
- Domain
- ci-cd, devops
- Issue type
- Refactor
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100