ClickHouse / ClickHouse/ClickHouse

ISSUE: IMPORT from s3 parquet with where statement (LESS ROWS COUNT INSERTED)

Open
#93,204 0 comments 0 reactions 0 assignees View on GitHub
comp-formats external potential bug
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

### Company or project name

_No response_

### Describe what's wrong

```
INSERT INTO logs_prod.logs (a_1,a_2) select (a_1 as a1, a_2 as a2)
FROM s3(
'https://prod.s3.ap-south-1.amazonaws.com/logs-parquet/logs-2025-12-*.parquet',
'Parquet',
extra_credentials(
role_arn = 'arn:aws:iam::xxxx:role/clickhouse-s3'
)
) ;
```

**this above query is working as expect but not with a where statement**

`where a2 = '111409077535327'; `

```
a2 is string in parquet
a_2 is FixedString(17) in logs
```

-- No dedup issue --
ENGINE = ReplacingMergeTree(a_1)
PARTITION BY toYYYYMM(a_1)
ORDER BY (a_2, a_1)
SETTINGS index_granularity = 8192;

### Does it reproduce on the most recent release?

No

### How to reproduce

STEP 1: CREATE TABLE
```
CREATE TABLE logs
(
`a_1` FixedString(17),
`a_2` DateTime64(3),
)
ENGINE = ReplacingMergeTree(a_1)
PARTITION BY toYYYYMM(a_1)
ORDER BY (a_2, a_1)
SETTINGS index_granularity = 8192;
```

STEP 2
IMPORT FROM S3 PARQUET WITH WHERE STATEMENT

```
INSERT INTO logs_prod.logs (a_1,a_2) SELECT (a_1 as a1, a_2 as a2)
FROM s3(
'https://prod.s3.ap-south-1.amazonaws.com/logs-parquet/logs-2025-12-*.parquet',
'Parquet',
extra_credentials(
role_arn = 'arn:aws:iam::xxxx:role/clickhouse-s3'
)
) where a2 = '111409077535327';
```

### Expected behavior

```
select count()
FROM s3(
'https://prod.s3.ap-south-1.amazonaws.com/logs-parquet/logs-2025-12-*.parquet',
'Parquet',
extra_credentials(
role_arn = 'arn:aws:iam::xxxx:role/clickhouse-s3'
)
) where a2 = '111409077535327';
```
// 200000 records

Insert should be 200000 records
** inseting only 15k records**

### Error message and/or stacktrace

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.