Altinity / Altinity/clickhouse-regression

Flaky tests in Altinity regression suite

Open
#94 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

cicd-failure
Dominant language
Python
Stars
27
Forks
10
Avg merge
1m
Merged PRs (30d)
2

Description

🧪 Description

This issue tracks flaky or unstable tests in the Altinity regression test suite.

These tests:

  • Are maintained by Altinity
  • Are not part of upstream ClickHouse tests
  • May fail intermittently but pass after reruns

⚠️ Flaky test List

  • Export Part

    • Test case: /s3/minio/export tests/export part/concurrent alter/during export/example=drop table
      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Intermittent
      • First observed: 2026-01-21
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
  • Swarms

    • Test case: /swarms/feature/swarm joins/join clause

      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Frequent
      • First observed: around 2025-12
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
    • Test case: /swarms/feature/feature/object storage cluster profile events with overloaded node

      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Frequent
      • First observed: around 2025-12
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
    • Test case: /swarms/feature/node failure/network failure

      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Frequent
      • First observed: around 2025-12
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
  • aggregate functions

    • Test case: /aggregate functions/part 3/merge/
      • Versions: 25.8.16.20002.altinityantalya
      • Flakiness: Rare
      • First observed: Mar 22, 2026
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:


📝 Instructions

Please follow the guidelines below when tracking new flaky tests in this issue.

➕ Adding a new flaky test

To add a new flaky test to the list, use the following template:

- **SUITE NAME**
  - Test case: `PATH/TO/TEST`
    - Versions: 
    - Flakiness: 
    - First observed: 
    - Assigned to: 
    - Status: Pending investigation
    - 🔗 Links:
      - CI run
      - Query in history database
📊 Flakiness levels

Use one of the following values for Flakiness:

  • Rare (< 5%)
  • Intermittent (< 30%)
  • Frequent (> 30%)

Choose the level based on how often the test fails in CI.

📌 Status values

Use one of the following Status values:

  • Pending investigation — flaky observed, not investigated yet
  • Under investigation — investigation in progress
  • Fixed (pending confirmation) — fix applied, awaiting CI stability
  • Fixed — confirmed stable after multiple CI runs
🔍 Investigation workflow

For deeper investigation:

  • Create a sub-issue under this issue for the flaky test
  • Use the sub-issue to track analysis, logs, discussions, and fixes

Once the flakiness is fixed and confirmed stable after multiple CI runs:

  • Update the sub-issue accordingly
  • Remove the test from the list in the description

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked CI runs and the history-database queries for the listed test paths, including the Export Part, Swarms, and aggregate functions cases. Create a sub-issue for one flaky test and use its logs and reruns to investigate. Done means the cause is addressed, stability is confirmed across multiple CI runs, and the entry is updated or removed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.