mattpocock / mattpocock/evalite
Display more trace details
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1.7k
- Forks
- 102
- PR merge metrics
- No merged PRs in 30d
Description
I'm testing an AI agent which reasons across multiple steps. The first thing I want to do when an eval fails is to see the chain of steps that were executed. I was expecting that to be what the "trace" view would do in the UI, but despite seeming teasingly clickable, it looks like it currently only shows the execution time for each trace (nothing happens when clicked on)
I'm working around this by incluiding the step logs as an additional property in the task result, so it shows up as a JSON column, but it's a bit clunky to have it always included, and I end up having to repeat myself on every step. Plus we could build a bespoke UI for displaying the trace contents that's much easier to read.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is identified in the issue. Start by examining the existing trace view in the UI and how task results expose step logs; determine how the executed chain is represented. Done means clicking a trace displays its steps in a readable bespoke UI rather than only execution time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, frontend, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100