mattpocock / mattpocock/evalite
Bug: Evalite Creates Cache Directory Regardless of `cacheEnabled` Setting
Open
@mattpocock is already working on this.
Since Nov 28, 2025.
- Dominant language
- TypeScript
- Stars
- 1.7k
- Forks
- 102
- PR merge metrics
- No merged PRs in 30d
Description
Description
Evalite unconditionally creates the cache directory at node_modules/.evalite/files/ during initialization, even when cacheEnabled: false is explicitly set in the configuration. This causes failures in read-only filesystem environments like AWS Lambda.
Root Cause
In run-evalite.js, the runEvalite() function creates the files directory before checking the cacheEnabled option:
export const runEvalite = async (opts) => {
const cwd = opts.cwd ?? process.cwd();
const filesLocation = path.join(cwd, FILES_LOCATION); // Line ~135
await mkdir(filesLocation, { recursive: true }); // Line ~136 - UNCONDITIONAL!
// ... 170+ lines later ...
const cacheEnabled = opts.cacheEnabled ?? config?.cache ?? true; // Line ~177
The FILES_LOCATION is hardcoded in backend-only-constants.js:
const CACHE_LOCATION = path.join("node_modules", ".evalite");
export const FILES_LOCATION = path.join(CACHE_LOCATION, "files");
Impact
This prevents evalite from running in environments with read-only filesystems, such as:
- AWS Lambda (where only
/tmpis writable) - Docker containers with read-only root filesystems
- CI/CD environments with restricted filesystem permissions
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.