diem / diem/diem-devtools

Define a protocol to communicate between individual tests and nextest

Open
#41 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
14
Forks
30
PR merge metrics
No merged PRs in 30d

Description

Right now the only communication channel we have between the testrunner and Rust tests, during execution, is the process exit code. We'd like to have a richer protocol but the Rust test harness is kind of limited so we need to be careful about it. Some ideas for things to communicate:
* test flakiness
* test retries
* test timeouts

Some ideas for the protocol itself:

* Use lines to stdout (or stderr) that begin with `nextest:` (similar to `cargo:` lines in build scripts)
* Pass in a random string through the environment and ensure that lines begin with that to avoid confusing random test output with nextest control lines.
* Publish a crate which can be used to output these lines, either as function calls or as a proc macro.

Contributor guide

Open the contributing guide

Research direction

The issue names no files or tests. Start by reviewing the Rust test harness and nextest execution boundary, then compare the proposed stdout/stderr, environment-token, and crate approaches; done means an agreed protocol that supports richer test-run communication.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
testing, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.