mattpocock / mattpocock/evalite
Don't log task results when using `evalite serve`
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1.7k
- Forks
- 102
- PR merge metrics
- No merged PRs in 30d
Description
Background
I have a fairly large eval suite (2.7k tasks). When running evalite serve I still get the tasks logged into the terminal, despite having the UI output.
Update: I realized after creating this that I think the --hideTable is supposed to do this in the beta, but it doesn't seem to be working. So perhaps this is more of a bug report than a feature request :)
Outcome
I think it would make sense when running with evalite serve to not log individual task results, given the purpose of the command is to view them in the UI.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the evalite serve command and its --hideTable handling in the TypeScript CLI entry points. Reproduce the issue with a large eval suite, then verify that individual task results no longer appear in the terminal while the UI still displays them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100