mattpocock / mattpocock/evalite

Add support for running multiple eval files by name

Open
#367 0 comments 4 reactions 1 assignee View on GitHub

@cantemizyurek is already working on this.

Since Dec 16, 2025.

enhancement
Dominant language
TypeScript
Stars
1.7k
Forks
102
PR merge metrics
No merged PRs in 30d

Description

Hi there! 👋

I'd love to be able to run a set of specific test files by file path in much the same way as I would with other test runners or eval runners:

evalite ./src/system-prompt.eval.ts ./src/tools/search.eval.ts

Vitest uses the interface of vitest [...filters], allowing you to provide file name patterns, like so:

vitest basic

Runs tests on the following files:

basic.test.ts
basic-foo.test.ts
basic/foo.test.ts

This interface also allows you to provide multiple filters, supporting multiple verbatim paths to be provided to Vitest.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.