Update the eval directory, implement more robust evaluation
Open
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 2
Description
Use Jaccard distance instead of binary pass/fail for each test?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by inspecting the eval directory and locating where each test is currently reduced to a binary pass/fail result. Determine how Jaccard distance should replace that result for each test; done means the evaluation reports the new distance-based outcomes consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100