lambdaisland / lambdaisland/kaocha-junit-xml

Reporting test cases marked with "pending" Kaocha tag as "skipped" in JUnit XML output

Open
#20 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Clojure
Stars
15
Forks
11
PR merge metrics
No merged PRs in 30d

Description

Background

Kaocha allows tests to be marked with a pending tag. This allows tests to be filtered when running tests using Kaocha but still have the data reported in the output. This is useful for building a suite of tests for a pending, yet-to-be-implemented feature. The following is an example of a simple test marked with this tag:

(ns sample.project.sample-test
  (:require [clojure.test :refer [deftest is]]))

(deftest ^:kaocha/pending pending-test  
  (is (= 1 1)  
      "This should be marked as 'skipped' by JUnit!"))

I am working in a Polylith codebase using polylith-kaocha as the test runner Running tests yield the following output:

1 test, 0 assertions, 1 pending, 0 failures.

PENDING sample.project.pending-test (:)

We've configured our polylith-kaocha tests to generate JUnit XML test results using this plugin in our Kaocha config. This is being generated however we're hitting a snag because the pending tests aren't actually running (as expected), but in the JUnit XML test results they look like they are running and passing:

  1. Run the sample-test suite
    clj -M:poly test
  2. Open the generated JUnit XML test report:
<?xml version='1.0' encoding='UTF-8'?>  
<testsuites>  
  <testsuite errors="0" package="" tests="1" name="sample-test" time="0.545124"    hostname="localhost" id="0" timestamp="2025-11-11T10:51:50" failures="0">  
    <properties/>  
    <testcase name="sample.project.sample-test/pending-test"         classname="sample.project.sample-test" time="0.000000" line="4" column="1" file="/sample/project/sample_test.clj">  
    </testcase>  
  <system-out/>  
  <system-err/>  
</testsuites>
  1. Note the pending-test result in the JUnit XML test report is missing the <skipped /> element that would usually be nested within <testcase> element to indicate that the test has been skipped.

Desired behaviour

  1. Run the sample-test suite
    clj -M:poly test
  2. Open the generated JUnit XML test report:
<?xml version='1.0' encoding='UTF-8'?>  
<testsuites>  
  <testsuite errors="0" package="" tests="1" name="sample-test" time="0.545124"    hostname="localhost" id="0" timestamp="2025-11-11T10:51:50" failures="0">  
    <properties/>  
    <testcase name="sample.project.sample-test/pending-test"         classname="sample.project.sample-test" time="0.000000" line="4" column="1" file="/sample/project/sample_test.clj">  
      <skipped/>
    </testcase>  
  <system-out/>  
  <system-err/>  
</testsuites>
  1. Observe the test result for sample-test is correctly tagged as skipped

Please let me know if I can provide any more detail, as this is my first time raising an issue on GitHub. Thanks in advance for any help you can provide!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the sample test marked ^:kaocha/pending and the clj -M:poly test command, then inspect the repository's JUnit XML reporting entry point and pending-result handling. Compare the generated testcase with the desired XML; done means pending tests include a <skipped/> element without being reported as passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.