ClickHouse / ClickHouse/ClickHouse
Data mismatch when reading via s3Cluster table function
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
### Company or project name
_No response_
### Describe what's wrong
A data mismatch occurs when exporting and re-importing data using the S3 table function with compression mode set to `xz` and read mode set to `auto` in a multi-shard cluster environment.
I get this issue with ClickHouse `25.8.10` but not with `25.8.4.13`
Cluster setup:
```xml
clickhouse1
9000
clickhouse2
9000
clickhouse3
9000
```
### Does it reproduce on the most recent release?
Yes
### How to reproduce
**ClickHouse version:** `25.8.10`
**ClickHouse settings:** `clickhouse client -n --allow_experimental_analyzer "0" --min_os_cpu_wait_time_ratio_to_throw "15" --max_os_cpu_wait_time_ratio_to_throw "25" --s3_truncate_on_insert "1"`
### 1. Create initial tables
```sql
CREATE TABLE table_1 (d UInt64)
ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='default';
```
```sql
CREATE TABLE table_2_local (d UInt64)
ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='default';
```
```sql
CREATE TABLE IF NOT EXISTS table_2 AS table_2_local
ENGINE = Distributed(cluster_3shards_1replicas_0, default, table_2_local, rand());
```
### 2. Insert test data
```sql
INSERT INTO table_1 VALUES (427);
```
### 3. Export data to S3 using compression `'xz'`
```sql
INSERT INTO FUNCTION s3('https://s3.amazonaws.com/.../auto.xz',
'key_id', 'access_key',
'CSVWithNames', 'd UInt64', 'xz')
SELECT * FROM table_1;
```
### 4. Re-import data from S3 with compression `'auto'`
```sql
INSERT INTO table_2
SELECT * FROM s3Cluster('cluster_3shards_1replicas_0',
'https://s3.amazonaws.com/.../auto.xz',
'key_id', 'access_key',
'CSVWithNames', 'd UInt64', 'auto');
```
### 5. Validate data
```sql
SELECT * FROM table_2 FORMAT CSV;
```
## Observed Result
The query returned an **empty result** instead of the expected value `427`.
## Expected Result
The imported table should contain the same data as the original (value `427`).
### Expected behavior
_No response_
### Error message and/or stacktrace
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.