boostorg / boostorg/regression

Server-side regression file processing is fragile and discourages community participation by design.

Open
#68 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
11
Forks
26
PR merge metrics
No merged PRs in 30d

Description

The mechanism of `default_filter_runners` and the `filter_runners` parameter to `list_ftp` in `reports/src/boost_wide_report.py` is fragile and discourages community participation (new testers) by design.

Fragility: changing the name of a test runner causes that test run to fail to report. With all that's involved in spinning up a new regression test fixture, this mode of failure is the last suspected and misleads the test runner.

Server-side troubleshooting is impossible for the regression tester, as both sufficient privileges and esoteric knowledge of server-side logging seem to be required.

Despite heroes like @sdarwin , the scarcity of attention given to regression testing leaves the regression matrices essentially permanently neglected.

One can easily imagine mischief uploads to the ftp server, and I appreciate efforts to counter this.
- I suggest markers in the upload file itself to establish authenticity if this is the concern, and community discussion on this topic as well.
- A technique to obscure the upload URL in the source code might be helpful, since github makes the source more available than in the olden days. With python you can do it on import with a technique limited only by your creativity and imagination. A fun puzzle.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.