exercism / exercism/zig-test-runner

Testing zig solutions is slow on Exercism

Open
#63 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Zig
Stars
2
Forks
6
Avg merge
3h 33m
Merged PRs (30d)
3

Description

https://github.com/exercism/zig-test-runner/commit/82b56afe82002cb6342222734b6bdeb8b665734c is a significant speedup for a `RUN zig test foo` command that's added at the bottom of the `Dockerfile`. But it doesn't seem to be a speedup in production.

Some possible short-term ways to improve:

1. Ensure that Zig can fully use the cache. Maybe some combination of:
- [x] Set `USER` (done in https://github.com/exercism/zig-test-runner/commit/b8b9d0221601474df58fd793d85d5fc5897fd837)
- [x] Create the cache later (done in https://github.com/exercism/zig-test-runner/commit/3f2c0c29aefb8fcf4049f80ba11ef801a9d01741)
- [x] Create cache via `bin/run.sh`, not `zig test` directly (done in https://github.com/exercism/zig-test-runner/commit/063642036853d548a3daa0fd53ecac53492942a7)
- [x] Specify `-target` (done in https://github.com/exercism/zig-test-runner/commit/b58066f952603c8d20bd61e3f61bba81a47415ec)
- [ ] Change some paths of solution files
- [ ] Look closer at the how Exercism run the container
- [ ] Set zig cache environment variables, or pass zig cache location options to `zig test`
1. Put the cache on a `tmpfs`.
1. Update Zig. Maybe commits like https://github.com/ziglang/zig/commit/020105d0dde614538a5839ede697e63a43bf6aa6 could help.

Longer term:

1. Use a bigger Zig cache.
1. Wait for https://www.github.com/ziglang/zig/issues/16270.
1. Use the x86_64 backend (currently marked as experimental).
1. Use wasm, like https://playground.zigtools.org/ and https://github.com/zigtools/playground. Exercism doesn't support this for now.
1. Wait for perf improvements in https://www.github.com/ziglang/zig/projects/6
1. Consider trying the C backend (no longer marked as experimental), and compiling with a fast C compiler. Note that the issue for generating tcc-compatible code ([ziglang/zig#13576](https://www.github.com/ziglang/zig/issues/13576)) is currently marked for Zig 1.1.0. Aside: the Nim test runner currently compiles Nim to C, and then uses tcc.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.