ClickHouse / ClickHouse/dbt-clickhouse

Improve test performance

Open
#532 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
362
Forks
177
Avg merge
2d 10h
Merged PRs (30d)
8

Description

Issue created to track changes we made to increase test times after the findings on https://github.com/ClickHouse/dbt-clickhouse/issues/499

Currently we have two types of test depending on where the CH cluster is located:
- `test_matrix.yml`: Uses a local CH cluster inside a docker-compose cluster. Currently we execute in parallel a matrix of tests with dimensions: CH version and Python version. One extra dimension is done serially in each execution: connection being done with HTTP or Native protocols. Total time per job is around 10 minutes. this is not an issue right now but any improvement would be welcome
- `test_cloud.yml`: Uses a CH cluster hosted in CH cloud. Currently these tests are not being used as we have found some problems that made these tests took too much time (>2h with a lot of flakyness):
- The CH Cloud cluster was already overloaded.
- The DBT test cannot be currently run in parallel against the same CH cloud instance as some test were colliding between them.

Pending improvements:
- Use a new CH Cloud cluster (we are working on it right now)
- Test a way to be able to execute the CH Cloud test in different clusters (also being worked)
- Be able to execute test in parallel: in progress, it will take time.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing test_matrix.yml and test_cloud.yml, along with the findings in issue 499, to understand the current matrix dimensions, cluster setup, and test collisions. Measure the existing test behavior before changing it; done means the cloud tests can use the new cluster setup reliably and run in parallel without collisions or excessive runtime.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, python
Domain
ci-cd, databases, performance, testing-qa
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.