LowCardinality INSERT corruption on retry/reuse of the same batch object after ambiguous failure
- Dominant language
- Go
- Stars
- 428
- Forks
- 84
- Avg merge
- 9d 6h
- Merged PRs (30d)
- 4
Description
### Describe the bug
We are seeing data corruption when retrying the same logical `INSERT` batch after an ambiguous failure such as `read timeout`, if the same batch / column objects are reused in `ch-go`.
The issue affects `LowCardinality(String)` columns and does not look like row loss:
- row count stays correct
- row keys stay correct
- `uniq(trace_id, span_id)` stays correct
- corruption is visible only in LC string metadata columns
- first insert attempt is clean, retry copies are corrupted
Observed pattern:
- expected `lg-000257` becomes `lg-000001`
- expected `lg-000258` becomes `lg-000002`
This points to a stale dictionary / key remapping problem around the `256` boundary.
At unit level this appears to come from `proto.ColLowCardinality.Prepare()` keeping internal dictionary state across repeated prepares / reuse.
### Steps to reproduce
1. Create a `LowCardinality(String)` column.
2. Fill it with 256+ unique values and call `Prepare()`.
3. Reuse the same column object for another logical batch, or call `Prepare()` again on the same object.
4. Encode and decode the column.
5. Observe mixed / shifted values.
### Expected behaviour
Repeated `Prepare()+Encode` on the same `LowCardinality` column object should be safe.
Reusing a batch object for retry should not preserve stale LC dictionary state from the previous logical batch.
### Code example
in [PR](https://github.com/ClickHouse/ch-go/pull/1156)
### Error log
Current failures:
- repeated Prepare() on 257 unique values decodes the last row as lg-000001 instead of lg-000257
- reuse with first batch lg-000001..lg-000256 and second batch lg-000257..lg-000512 decodes second batch as lg-000001..lg-000256
- key width selection around 255/256/257 is also suspicious
### Configuration
#### Environment
- Client version: ch-go v0.71.0
- Language version: Go
- OS: ubuntu-noble
#### ClickHouse server
- ClickHouse Server version: 25.3, 25.8
- CREATE TABLE statements for tables involved:
```sql
CREATE TABLE traces
(
trace_id UUID,
span_id UInt64,
tenant LowCardinality(String),
service LowCardinality(String),
deployment LowCardinality(String),
operation LowCardinality(String)
)
ENGINE = MergeTree
ORDER BY (trace_id, span_id)
```
- Sample data for all these tables:
Minimal synthetic pattern:
- first batch: lg-000001..lg-000256
- second batch: lg-000257..lg-000512
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.