Altinity / Altinity/clickhouse-regression
Feature-support validation: fail fast on missing Antalya features on antalya-26.6 (swarms, oauth, s3 export, iceberg)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 27
- Forks
- 10
- Avg merge
- 1m
- Merged PRs (30d)
- 2
Description
This issue records work done while running regression suites against the new antalya-26.6 base. Antalya features are forward-ported to a new base at different times, so several suites failed en masse because the features they test are not present yet. The work was consolidated into a single commit, plus two follow-up commits (see below). Assisted by AI (Cursor / Claude Opus 4.8).
Summary
On antalya-26.6, suites that exercise not-yet-forward-ported Antalya features produced large numbers of unrelated failures (e.g. swarms had ~3000+ failures, all rooted in one missing setting), which makes triage hard and hides real regressions.
Added a reusable feature-support validation so each affected suite/sub-feature emits one clear failure (a feature support validation subtest) instead of many, and does not run the rest once the feature is known to be missing. Gates are antalya-only where applicable, so non-antalya behavior is unchanged, and they auto-recover once the feature is forward-ported (the check passes and the tests run normally).
Fix commit: fb33c1f4c
Generic component
helpers/feature_support.py:
feature_support_validation— a@TestScenarionamedfeature support validationthat fails with a clear message when the feature is absent.validate_feature_support(self, feature, check)— runs the detection on the module context and the named subtest; returns the support bool so a suite canreturn/skip the rest.setting_supported(name)— check that a setting exists (queries thenamecolumn ofsystem.settings, not itsvalue— several settings default to an empty string, so a value-based check gives false negatives).get_clickhouse_binary/run_clickhouse_local— pre-cluster host probe viaclickhouse local, for features that stop the server from starting.
Per-suite gates
| Suite / sub-feature | Detection | Notes |
|---|---|---|
swarms |
object_storage_cluster setting |
collapsed ~3000+ failures into one |
oauth |
CREATE USER ... IDENTIFIED WITH jwt via clickhouse local (pre-cluster) |
server can't even start without JWT (static jwt_user in users.xml is rejected at boot) |
s3 export part |
allow_experimental_export_merge_tree_part |
|
s3 export partition |
export_merge_tree_partition_force_export |
partition-specific, not the general export-part setting |
iceberg swarm |
object_storage_cluster |
|
iceberg iterator race condition |
object_storage_cluster_join_mode |
|
iceberg sort key timezone |
iceberg_partition_timezone |
|
iceberg export partition |
export_merge_tree_partition_force_export |
server crashes on config restart without it; gate runs before applying the config |
iceberg system.tables partition/sorting keys |
capability probe (partition_key populated in system.tables) |
PR Altinity/ClickHouse#1432 — no setting exists, so we probe the actual capability |
Detection notes
- Most features are gated by a setting existence check (post-cluster query, or pre-cluster
clickhouse localfor oauth). system.tables partition/sorting keyshas no setting (compiled-in behavior from PR #1432, "Antalya 26.1"), so it uses a capability probe; the test's own version gate (>26.3.10.20001) wrongly assumed presence on 26.6.- Confirmed detection empirically against a build that has the features (
26.3.10.20001.altinityantalya, passes) vsantalya-26.6(fails fast).
Follow-up commits
6933926130— all setting-based gates now callsetting_supported()directly. Removes three duplicate wrappers (export_merge_tree_part_supported,export_merge_tree_partition_supported,object_storage_cluster_supported) andswarms/tests/feature_support.py. From Julian's review: the same check forexport_merge_tree_partition_force_exporthad been written two different ways across the s3 and iceberg suites.867b4caf2f— fixes aTypeErrorthis commit introduced inswarms/regression.py.validate_feature_support()ends with aScenario, so thewith And(...)below it had noStepsibling to inherit its subtype from. It only triggers when the gate passes, so the swarms suite did not actually run on a swarm-capable build until this fix — the "tests run normally" part of the summary above only holds from867b4caf2fon.
Follow-ups (not addressed here)
- None
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The described work is already implemented across helpers/feature_support.py and the swarms, oauth, s3, and iceberg suites; start by reviewing commits fb33c1f4c, 6933926130, and 867b4caf2f. Run the affected regression suites against a feature-capable build and antalya-26.6 to verify that missing features produce one validation failure while supported features run normally.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100