citusdata / citusdata/test-automation
Random Order for Testing 2 Different Versions
- Dominant language
- C
- Stars
- 15
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
Sometimes there are factors in our machines that can slow down performance a bit. It might be after a certain point. For example with AWS we have seen that IOPS would drop after some credits is reached.
We currently do not know anything like that for Azure, but there could be some 'yet-unknown' factors. I guess to be on the safe side, it would make sense to choose the order of versions randomly. So currently if we always run the later version first. We can decide that randomly so that this could bring us more confidence about performance. If a version is faster than another in both cases(running first and last) then it is very likely that it is faster.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.