Test-More / Test-More/Test2-Harness
Additional control over test order
Nobody has claimed this yet.
- Dominant language
- Perl
- Stars
- 28
- Forks
- 30
- PR merge metrics
- No merged PRs in 30d
Description
I have been having a conversion via email about a desired feature. The requester has a very specific test setup where some things must run in a specific order, with other tests running parallel between them.
Currently they use the following yaml file to specify the run order when using prove.
# The wilrcards below ARE NOT regexes, they are globs and
# specific to TAP::Parser::Scheduler. The only possibilities are:
# ** is any number of characters, including /, within a pathname
# * is zero or more characters within a filename/directory name
# ? is exactly one character within a filename/directory name
# {foo,bar,baz} is any of foo, bar or baz.
# \ is an escape character
seq:
- seq:
- par:
- t/*
- xt/{0,1,2,3}*
- seq:
- xt/40-dbsetup.t
- par:
- xt/40-database.t
- xt/41-coaload.t
- xt/45*.t
- xt/47*.t
- xt/48*.t
- seq:
- par:
- xt/{43,44,5,7}*.t
- xt/6{1,2,3,7,8,9}*.t
- seq:
- xt/60-startlsmb.t
- seq:
- par:
- xt/6{5,6}*.t
- seq: **.feature
- xt/*.pg
- seq: xt/89-dropdb.t
Achieving this behavior requires 2 things:
- A custom Test2::Harness::Finder subclass
- Custom behavior for Test2::Harness::Runner::State::advance_tasks()
I think a new plugin RulesYAML should be added, it can replace the default finder simply enough. Currently there is no way to specify a custom State subclass, but that functionality should not be difficult.
https://metacpan.org/pod/TAP::Parser::Scheduler has a description of the rules used when parsing the yaml file.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Test2::Harness::Finder, Test2::Harness::Runner::State::advance_tasks(), and the TAP::Parser::Scheduler rules linked in the issue. Trace how prove selects and advances tasks, then determine how a RulesYAML plugin and configurable State subclass should support sequential and parallel groups; done means the YAML-described ordering works through the harness.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- perl
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100