Testing suite roadmap
- Dominant language
- Nim
- Stars
- 32
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
There has been great progress in building up the test part of the test suite, however the software running the tests is quite lacking.
A replacement should be developed that fulfills the following criteria.
- [x] imports src/main.nim instead of running pre-built binaries
- [x] Runs tests parallel (multiple instances of japl)
- [x] Separates stdout/stderr output
With this, a more advanced testing language is needed.
- [x] Replace output: with stdout: stderr: (and stdin:)
- [x] Allow for multiple tests per file
- [x] Allow regex (or peg) matches (handy for exception messages)
The visual of the test runner also needs improvement:
- [x] Add a progress bar indicating test progress
- [x] Improve the CLI (flags for verbosity and such)
Long term improvements:
- [ ] Allow writing a parallel test in python (checks if output matches)
- [ ] Benchmark support
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading src/main.nim and reviewing the testing-suite roadmap, including the completed runner requirements and the remaining Python parallel-test and benchmark items. Done would require a defined implementation scope for those long-term improvements, but the issue does not identify runner files or tests to run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nim, python
- Domain
- testing, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100