microsoft / microsoft/testfx

Message for Notifying about Hooks/Non-Tests

Open
#4,087 22 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/mtp needs/design
Dominant language
C#
Stars
1k
Forks
312
Avg merge
8h 30m
Merged PRs (30d)
469

Description

## Summary

A new message that can publish information about hooks/Non-Tests

## Background and Motivation

Currently if a hook fails, I have two options:
- attach the failure to an existing test node
- create a dummy test node to represent the hook

Those problems with these are, respectively:
- this implies the test failed which isn't the case, and it also complicates code bases more as the test object must be kept around for longer, and also timings. Do I count hook timings in the test that triggered it? Ideally no, it would have its own object with it's own timing.
- dummy test nodes are reported as new tests, and would then show under the test counters at the end, and IDEs would show them as runnable tests.

## Proposed Feature

A new message to represent hooks would be nice.

It would simplify my logic as I wouldn't have to keep test objects around for longer and delay their result publishing until hooks had finished.

Optionally, supplying a property of test IDs to them could represent what tests they relate to/trigger from. These could be used to drive IDE uis if they wanted to build something to show hook execution.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by reviewing the existing message model and hook-result publishing path, then define the protocol and acceptance criteria for a hook message, including whether related test IDs are required; done means hooks are represented without appearing as failed or runnable tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.