citusdata / citusdata/test-automation

Random Order for Testing 2 Different Versions

Open
#131 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C
Stars
15
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Sometimes there are factors in our machines that can slow down performance a bit. It might be after a certain point. For example with AWS we have seen that IOPS would drop after some credits is reached.

We currently do not know anything like that for Azure, but there could be some 'yet-unknown' factors. I guess to be on the safe side, it would make sense to choose the order of versions randomly. So currently if we always run the later version first. We can decide that randomly so that this could bring us more confidence about performance. If a version is faster than another in both cases(running first and last) then it is very likely that it is faster.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.