No test events generated for Project elements
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 234
- Forks
- 163
- Avg merge
- 13h 32m
- Merged PRs (30d)
- 25
Description
PR #596 (currently awaiting merge) creates Project test-suite elements (i.e. tests) for each project being run, with one or more Assembly test-suites as children of the project.
This solves the problem of the absence of project elements in the result XML and in displays constructed by any GUI that shows the entire test tree (e.g. TestCentric) but creates one additional, albeit minor, problem.
Virtually all test events are generated by the framework itself. Currently, the only exception is the start of run and end of run events, which are generated by the engine. In principle, any test nodes created by the engine should have test start and end events corresponding to them.
The lack of these events is only visible in a GUI that shows the entire tree of tests. In TestCentric GUI, for example, the tree node representing the project is never flagged as passed, failed or even run because no event is ever received. The framework, of course, can't generate such events, since it knows nothing about projects. The highest level test for which the framework generates start and finish events is the assembly.
This could be handled in the MasterTestRunner, which is currently the only runner that knows about projects, in one of two ways:
- Special handling for project nodes.
- Special handling for Engine-created nodes (currently only test-run and project)
The latter is more general and may require a new level of abstraction to deal with engine-created tests.
In either case, what needs to be done is
-
Note which projects contain which assemblies. This info is available in both the TestPackage and the constructed tree of tests. It may need to be made more accessible through some sort of assembly to containing project lookup.
-
Intercept and examine events from the framework before passing them on to the listeners.
-
When the first assembly for a project starts, issue the project start event before passing on that for the assembly.
-
When the last assembly for a project finishes, issue the project end event after passing on that for the assembly.
This could have been done as part of #596 but seems a bit complicated such a small problem, so I decided to make it a separate issue. Currently, the only side effect is that the project tree nodes remain grey rather than changing color and it's probably possible to build a workaround into the GUI until this is fixed in the engine.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in MasterTestRunner and trace how TestPackage project-to-assembly information and framework events are forwarded to listeners. Determine how to identify the first assembly starting and last assembly finishing for each project. Done means project start and end events surround the relevant assembly events so project nodes receive status in the result tree.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100