e2e test does not fail even if the task actually failed
Open
Nobody has claimed this yet.
kelos/needs-input
kind/bug
needs-actor
priority/important-soon
triage-accepted
- Dominant language
- Go
- Stars
- 331
- Forks
- 40
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 70
Description
e2e test does not fail even if the task to write a file failed. The agent explains the reason for not being able to do that, but we cannot differentiate it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating and running the e2e test that writes a file, then trace how task failures are reported when the agent explains it could not complete the task. Done means the e2e test reliably distinguishes a successful file-writing task from one that failed, rather than passing in both cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100