Regression testing for examples
- Dominant language
- C++
- Stars
- 245
- Forks
- 36
- Avg merge
- 8d 23h
- Merged PRs (30d)
- 4
Description
We should create some mechanism to track changes of example results over time.
https://en.wikipedia.org/wiki/Regression_testing
It may be possible to track result values through adiak, and upload the examples similar to how we do with benchmarks.
```cpp
SERAC_SET_METADATA("result", result);
```
Using SPOT or Hatchet, we can then graph change in this value over time.
We should also see how other projects do regression testing or something similar.
Contributor guide
Research direction
No files or tests are named. Start by reviewing how examples produce results and how benchmarks are uploaded, then investigate whether adiak can track result values and how SPOT or Hatchet could graph them. Done means there is an agreed mechanism for tracking example-result changes over time, informed by comparable projects.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100