citusdata / citusdata/citus

Split Tests

Open
#5,539 0 comments 0 reactions 0 assignees View on GitHub
regression tests
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

It seems that `check-isolation` is one of the bottlenecks in our test suite in terms of the run time and it should be easy to split isolation tests: https://app.circleci.com/insights/github/citusdata/citus/workflows/build_and_test/jobs?branch=b1d8acd4-8701-4c76-8a87-7ff2320aa95f&reporting-window=last-30-days

A more Circleci way of splitting the tests would be the following:(which is much more work but more ideal and long term solution, for the whole test suite, not just check-isolation):
- We modify the tests so that they pass when run as a single test on top of check-base
- At this point, each test file can be run separately hence we use `circleci tests split` similar to https://github.com/citusdata/citus/blob/7d0f4f11c30a0de0f77041480a704185f179103a/.circleci/config.yml#L233.
- Circleci will then scatter the tests around containers and increasing the parallelism will be a single line change.

(Circleci can also scatter the tests by runtime, so it would be quite good in our case)

The above would help us:
- Decrease the tests times a lot
- Keep the tests time stable (my increasing the number of containers when they increase)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.