[Improvement] Support ClickHouse data-skipping indexes and engine-aware sort orders validation in CreateTableDialog
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
## Description
The `CreateTableDialog` in the Gravitino web UI needs two enhancements for ClickHouse table creation:
### 1. Support ClickHouse data-skipping index types with properties
ClickHouse supports data-skipping indexes (`data_skipping_minmax`, `data_skipping_bloom_filter`, `data_skipping_set`) which have additional properties:
- **Granularity** - applies to all data-skipping index types
- **Set Max Values** - applies only to `data_skipping_set`
Currently, the Indexes tab does not render these property fields. The table creation form should:
- Show a **Granularity** column for data-skipping indexes (hidden for `primary_key`)
- Show a **Set Max Values** column only for `data_skipping_set` index type
- Populate these properties when editing an existing table
- Include index properties in the submit payload
### 2. Engine-aware sort orders validation
Sort orders are required for ClickHouse MergeTree family engines (`MergeTree`, `ReplacingMergeTree`, `SummingMergeTree`, etc.), but **not** required for other ClickHouse engine types (`Distributed`, `TinyLog`, `Log`, `StripeLog`).
Currently, the sort orders validation does not adapt when the user switches the engine type. This causes:
- Sort Orders tab always shows the required indicator (`*`) regardless of engine type
- Form validation fails for non-MergeTree engines when sort orders are empty
- Switching from a MergeTree engine to a non-MergeTree engine does not clear residual sort orders data or validation errors
The expected behavior:
- **MergeTree family engines**: Sort Orders tab shows `*` indicator, sort orders are required, nullable columns cannot be used in ORDER BY
- **Non-MergeTree engines** (Distributed, TinyLog, Log, StripeLog): Sort Orders tab does not show `*` indicator, sort orders are optional
- **Engine switch**: When switching from MergeTree to non-MergeTree, clear sort orders data and validation errors; when switching to MergeTree, show required validation if sort orders are empty
## Affected files
- `web-v2/web/src/app/catalogs/rightContent/CreateTableDialog.js`
- `web-v2/web/src/config/index.js` (already has `clickHouseMergeTreeEngines` and `indexesInfoMap` for `jdbc-clickhouse`)
## Implementation notes
- Sort orders validation is moved from `Form.Item rules` (which can cache stale closure values) to `handleSubmit` where `form.getFieldValue('engine')` is called to read the current engine value in real-time
- A `useEffect` on `isClickHouseMergeTreeEngine` handles real-time UI updates when the engine type changes
- Data-skipping index properties (granularity, set_max_values) are conditionally rendered using `Form.Item shouldUpdate` and persisted in the index `properties` object
Contributor guide
Assessment
This issue has not been assessed yet.