lambdaisland / lambdaisland/kaocha
Make a distinction between load errors in the test file being loaded vs. in other namespaces.
Nobody has claimed this yet.
- Dominant language
- Clojure
- Stars
- 860
- Forks
- 81
- PR merge metrics
- No merged PRs in 30d
Description
If an exception occurs while Kaocha loads a test namespace, then a warning is printed, the :kaocha.type/namespace test is marked as failed, and loading/running continues.
This is so that when you have a typo which prevents a test namespace from loading, this doesn't stop the world. Instead it just runs the rest of the tests, and reports a failure pointing at the syntax error.
The problem with this approach is that loading a namespace could fail because a dependent namespace has issues. In the degenerate case a source namespace that is used indirectly in every test fails to load, causing a failure to be reported for every test namespace. This is especially easy to do (and annoying to encounter) in --watch mode.
Instead when loading a test namespace we should first evaluate its ns form. If that causes issues then we hard abort the test run. Only after the ns form has loaded do we load the full namespace, and turn load errors into test failures.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing Kaocha's test-namespace loading path and the :kaocha.type/namespace test behavior. Check how the namespace's ns form is evaluated separately from the remaining forms, then verify that dependency load errors abort while test-namespace load errors become failures. Done means the distinction works in normal and --watch runs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clojure
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100