exercism / exercism/zig-test-runner
Testing zig solutions is slow on Exercism
- Dominant language
- Zig
- Stars
- 2
- Forks
- 6
- Avg merge
- 3h 33m
- Merged PRs (30d)
- 3
Description
https://github.com/exercism/zig-test-runner/commit/82b56afe82002cb6342222734b6bdeb8b665734c is a significant speedup for a `RUN zig test foo` command that's added at the bottom of the `Dockerfile`. But it doesn't seem to be a speedup in production.
Some possible short-term ways to improve:
1. Ensure that Zig can fully use the cache. Maybe some combination of:
- [x] Set `USER` (done in https://github.com/exercism/zig-test-runner/commit/b8b9d0221601474df58fd793d85d5fc5897fd837)
- [x] Create the cache later (done in https://github.com/exercism/zig-test-runner/commit/3f2c0c29aefb8fcf4049f80ba11ef801a9d01741)
- [x] Create cache via `bin/run.sh`, not `zig test` directly (done in https://github.com/exercism/zig-test-runner/commit/063642036853d548a3daa0fd53ecac53492942a7)
- [x] Specify `-target` (done in https://github.com/exercism/zig-test-runner/commit/b58066f952603c8d20bd61e3f61bba81a47415ec)
- [ ] Change some paths of solution files
- [ ] Look closer at the how Exercism run the container
- [ ] Set zig cache environment variables, or pass zig cache location options to `zig test`
1. Put the cache on a `tmpfs`.
1. Update Zig. Maybe commits like https://github.com/ziglang/zig/commit/020105d0dde614538a5839ede697e63a43bf6aa6 could help.
Longer term:
1. Use a bigger Zig cache.
1. Wait for https://www.github.com/ziglang/zig/issues/16270.
1. Use the x86_64 backend (currently marked as experimental).
1. Use wasm, like https://playground.zigtools.org/ and https://github.com/zigtools/playground. Exercism doesn't support this for now.
1. Wait for perf improvements in https://www.github.com/ziglang/zig/projects/6
1. Consider trying the C backend (no longer marked as experimental), and compiling with a fast C compiler. Note that the issue for generating tcc-compatible code ([ziglang/zig#13576](https://www.github.com/ziglang/zig/issues/13576)) is currently marked for Zig 1.1.0. Aside: the Nim test runner currently compiles Nim to C, and then uses tcc.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.