Improve reporting for passing "todo" tests

Open
#1,813 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
javascript
Domain
cli, testing

Research direction

Start by tracing how the CLI, TAP, and HTML Reporter produce the passing todo-test output shown in the issue. Compare each reporter with the expected examples; done means all three identify the TODO test, explain its failure status, and suggest removing the TODO marker when appropriate.

Written by the indexing model from the issue text.

Description

Component: Core Type: Enhancement

Status quo

When a todo test is failing, we currently report it as follows in the CLI and in TAP:

/* global QUnit */
QUnit.test('apple', function (assert) {
  assert.true(true, 'stem');
  assert.true(true, 'skin');
});
QUnit.test.todo('banana', function (assert) {
  assert.true(true, 'foo');
  assert.equal('This is actual.', 'This is expected.', 'example sentence');
  assert.true(true, 'bar');
  assert.true(true, 'baz');
});
QUnit.test('sauerkraut', function (assert) {
  assert.true(true, 'acid rock');
});
QUnit.test('dampfnudel', function (assert) {
  assert.true(true, 'stream');
  assert.true(true, 'bun');
});

TAP version 13
ok 1 apple
not ok 2 banana # TODO
  ---
  message: example sentence
  severity: todo
  actual  : This is actual.
  expected: This is expected.
  stack: at test/fixtures/qunit-todo-skip.js:8:10
  ...
ok 3 sauerkraut
ok 4 dampfnudel
1..4
# pass 3
# skip 0
# todo 1
# fail 0

... and in the HTML Reporter:

Default (collapsed, because not failing) Expected
Image Image

Problem

Then, when it starts passing, it is reported as folllows.

  • There is nothing here to indicate that it is/was a TODO test.
  • There is nothing here to explain why it is marked as failure.
  • There is nothing here to suggest removing the "TODO" marker to make it pass.
TAP version 13
ok 1 apple
not ok 2 banana
ok 3 sauerkraut
ok 4 dampfnudel
1..4
# pass 3
# skip 0
# todo 0
# fail 1
Image
Dominant language
JavaScript
Stars
4k
Forks
773
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from qunitjs/qunit

All issues in qunitjs/qunit

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.