DynamoRIO / DynamoRIO/dynamorio
add binary package automated testing
- Dominant language
- C
- Stars
- 3.2k
- Forks
- 629
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 31
Description
The testing we perform prior to a release (see
https://github.com/DynamoRIO/dynamorio/wiki/New-Release) consists mainly of
sanity checks running all of the samples and polished tools on both small
console apps but also on graphical apps like notepad. It would be nice to
automate that.
Xref adding graphical app testing to the pre-commit suite: #120.
This issue is about adding binary package automated testing, as opposed to
the source code building and testing of our regular test suite. The goal
here is to have a script that can be run on VM's that do not necessarily
have a development environment set up, for sanity testing on a wide range
of platforms.
Step 1 is to avoid Windows popups by making SHOW_RESULTS a runtime option
(separately but with the same name in each sample I suppose). Even if the
rest of the automation isn't finished, that alone would make testing
easier.
Step 2 is to take the current sequences of commands and turn them into a
more formal script with some automated checking of results.
Step 3 is to set up cron jobs to run in virtual machines periodically.
Perhaps when each "nightly build" is created: #1967?
We do not expect to be able to easily automate some pieces:
- .msi install and uninstall (for Dr. Memory)
- Visual Studio external tool testing (for Dr. Memory)
- admin -syswide_on testing
- cmd interactive output testing (less of an issue on recent Windows)
Contributor guide
Assessment
This issue has not been assessed yet.