lambdaisland / lambdaisland/kaocha
Could we store the test result in the results that are passed to the post-summary hook?
Nobody has claimed this yet.
- Dominant language
- Clojure
- Stars
- 860
- Forks
- 81
- PR merge metrics
- No merged PRs in 30d
Description
Right now, at least with version v1, the result of each test received by the post-summary hook contains the following keys
(:kaocha.result/error
:kaocha.result/pending
:kaocha.result/pass
:kaocha.var/test
:kaocha.plugin.randomize/sort-key
:kaocha.testable/events
:kaocha.plugin.capture-output/output
:kaocha.testable/meta
:kaocha.result/count
:kaocha.testable/type
:kaocha.var/name
:kaocha.var/var
:kaocha.testable/desc
:kaocha.testable/wrap
:kaocha.result/fail
:kaocha.testable/id)
I would like to have access to the test result, assuming kaocha is calling the function under :test we would be storing true for this simple case
(deftest simple-test
(is (= 1 1)))
((:test (meta #'simple-test)))
; => true
However, if you run a test using the defflow macro from state-flow the result is a pair [result state] which would allow me to store information in the state that I can later use to build a report in the post-summary hook.
(defflow my-flow {:init (constantly {:value 1
:map {:a 1 :b 2}})}
[value (state/gets :value)]
(testing "1" (mc/match? 1 value))
(testing "b is 2" (mc/match? {:b 2} (state/gets :map))))
((:test (meta #'my-flow)))
; => [{:match/result :match,
; :match/expected {:b 2},
; :match/actual {:a 1, :b 2},
; :probe/results [{:check-result true, :value {:a 1, :b 2}}],
; :probe/sleep-time 200,
; :probe/times-to-try 1}
; {:value 1, :map {:a 1, :b 2}}]
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the post-summary hook and trace how each test result is assembled from the function under the :test key. Determine where the returned value could be retained without breaking existing result data, then add coverage showing that both a boolean result and a [result state] value are available to the hook.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clojure
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100