lambdaisland / lambdaisland/kaocha

Support for test-ns-hook

Open
#29 11 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature
Dominant language
Clojure
Stars
860
Forks
81
PR merge metrics
No merged PRs in 30d

Description

clojure.test has this feature where you can define a function named test-ns-hook, and instead of running the test vars in the current namespace, it will run that function.

My impression is that this is mostly used if you want to run your test vars in a very specific order, which in turn is something that's not often needed (and generally a smell).

Kaocha currently does not support this. The problem with it is that it makes things opaque, we lose all the rich information in the test plan regarding test vars, so e.g. filtering will not work.

The easiest way to make this work is to treat the test-ns-hook var as a single test var, and ignore any other test vars in that namespace. This should work, but it may produce sub-optimal results. e.g. say you have this code

(deftest aaa)
(deftest bbb)

(defn test-ns-hook []
  (aaa)
  (bbb))

These calls to (aaa) and (bbb) go through clojure.test/test-var, rather than through Kaocha's testable machinery, meaning events will not have the right :kaocha/testable associated with them. (this in a way makes sense, because there will be no testable for aaa or bbb, since they are ignored in favor of test-ns-hook).

There are probably (hacky) ways around this, but given how little this seems to be used I intend to start with the simple approach, so at least we have some (albeit suboptimal) support for this.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing Kaocha's test discovery and testable machinery, then compare it with clojure.test's test-ns-hook behavior. Support a namespace defining test-ns-hook as one test var while ignoring other vars in that namespace. Done means such namespaces run under Kaocha, with the documented limitation that nested test events may lack the expected :kaocha/testable association.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.