ClickHouse / ClickHouse/ClickHouse

Data mismatch when reading via s3Cluster table function

Open
#88,699 0 comments 0 reactions 0 assignees View on GitHub
comp-codecs external potential bug
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

### Company or project name

_No response_

### Describe what's wrong

A data mismatch occurs when exporting and re-importing data using the S3 table function with compression mode set to `xz` and read mode set to `auto` in a multi-shard cluster environment.

I get this issue with ClickHouse `25.8.10` but not with `25.8.4.13`

Cluster setup:

```xml



clickhouse1
9000


clickhouse2
9000


clickhouse3
9000



```

### Does it reproduce on the most recent release?

Yes

### How to reproduce

**ClickHouse version:** `25.8.10`
**ClickHouse settings:** `clickhouse client -n --allow_experimental_analyzer "0" --min_os_cpu_wait_time_ratio_to_throw "15" --max_os_cpu_wait_time_ratio_to_throw "25" --s3_truncate_on_insert "1"`

### 1. Create initial tables
```sql
CREATE TABLE table_1 (d UInt64)
ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='default';
```

```sql
CREATE TABLE table_2_local (d UInt64)
ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='default';
```

```sql
CREATE TABLE IF NOT EXISTS table_2 AS table_2_local
ENGINE = Distributed(cluster_3shards_1replicas_0, default, table_2_local, rand());
```

### 2. Insert test data
```sql
INSERT INTO table_1 VALUES (427);
```

### 3. Export data to S3 using compression `'xz'`
```sql
INSERT INTO FUNCTION s3('https://s3.amazonaws.com/.../auto.xz',
'key_id', 'access_key',
'CSVWithNames', 'd UInt64', 'xz')
SELECT * FROM table_1;
```

### 4. Re-import data from S3 with compression `'auto'`
```sql
INSERT INTO table_2
SELECT * FROM s3Cluster('cluster_3shards_1replicas_0',
'https://s3.amazonaws.com/.../auto.xz',
'key_id', 'access_key',
'CSVWithNames', 'd UInt64', 'auto');
```

### 5. Validate data
```sql
SELECT * FROM table_2 FORMAT CSV;
```

## Observed Result
The query returned an **empty result** instead of the expected value `427`.

## Expected Result
The imported table should contain the same data as the original (value `427`).

### Expected behavior

_No response_

### Error message and/or stacktrace

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.