japl-lang / japl-lang/japl

Testing suite roadmap

Open
#35 1 comment 0 reactions 0 assignees View on GitHub
enhancement good first issue help wanted
Dominant language
Nim
Stars
32
Forks
1
PR merge metrics
No merged PRs in 30d

Description

There has been great progress in building up the test part of the test suite, however the software running the tests is quite lacking.
A replacement should be developed that fulfills the following criteria.
- [x] imports src/main.nim instead of running pre-built binaries
- [x] Runs tests parallel (multiple instances of japl)
- [x] Separates stdout/stderr output

With this, a more advanced testing language is needed.
- [x] Replace output: with stdout: stderr: (and stdin:)
- [x] Allow for multiple tests per file
- [x] Allow regex (or peg) matches (handy for exception messages)

The visual of the test runner also needs improvement:
- [x] Add a progress bar indicating test progress
- [x] Improve the CLI (flags for verbosity and such)

Long term improvements:
- [ ] Allow writing a parallel test in python (checks if output matches)
- [ ] Benchmark support

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading src/main.nim and reviewing the testing-suite roadmap, including the completed runner requirements and the remaining Python parallel-test and benchmark items. Done would require a defined implementation scope for those long-term improvements, but the issue does not identify runner files or tests to run.

Written by the indexing model from the issue text.

Assessment

Tech stack
nim, python
Domain
testing, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.