ClickHouse / ClickHouse/clickhouse-go
Unexpected behavior when scanning `Nullable(Decimal64(2))` column
- Dominant language
- Go
- Stars
- 3.3k
- Forks
- 684
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 14
Description
## Observed
1. Create a table with a single `Nullable(Decimal64(2))`
2. Insert `(NULL), (NULL), (NULL), (123.1), (NULL), (124.1), (NULL), (NULL), (NULL), (NULL)`
3. Read back the rows and observe the pointer not being reset when encountering a null value. The previous row's value is returned instead.
4. `NULL, NULL, NULL, 123.1, 123.1, 124.1, 124.1, 124.1, 124.1, 124.1` returned
## Expected behaviour
The pointer should be reset when encountering `NULL` and not return the previous row's value.
## Code example
Reproduction [example from the original issue](https://github.com/ClickHouse/clickhouse-go/issues/955#issuecomment-1493676932) modified for `decimal.Decimal`.
`examples/clickhouse_api/dynamic_scan_types.go`
```go
package clickhouse_api
import (
"context"
"fmt"
"reflect"
"github.com/shopspring/decimal"
)
func DynamicScan() error {
conn, err := GetNativeConnection(nil, nil, nil)
if err != nil {
return err
}
if err := conn.Exec(context.Background(), "CREATE TABLE IF NOT EXISTS dynamic_scan_table (Col1 Nullable(Decimal64(2))) ENGINE = Memory"); err != nil {
return err
}
if err := conn.Exec(context.Background(), "INSERT INTO dynamic_scan_table VALUES (NULL), (NULL), (NULL), (123.1), (NULL), (124.1), (NULL), (NULL), (NULL), (NULL)"); err != nil {
return err
}
rows, err := conn.Query(context.Background(), "SELECT Col1 FROM dynamic_scan_table")
if err != nil {
return err
}
var (
columnTypes = rows.ColumnTypes()
vars = make([]interface{}, len(columnTypes))
)
for i := range columnTypes {
vars[i] = reflect.New(columnTypes[i].ScanType()).Interface()
}
for rows.Next() {
if err := rows.Scan(vars...); err != nil {
return err
}
for _, v := range vars {
switch v := v.(type) {
case **decimal.Decimal:
if *v == nil {
fmt.Println("NULL")
continue
}
fmt.Println(**v)
}
}
}
return nil
}
```
```
$ go test ./... -v -run DynamicScan
...
=== RUN TestDynamicScan
NULL
NULL
NULL
123.1
123.1
124.1
124.1
124.1
124.1
124.1
--- PASS: TestDynamicScan (0.06s)
```
## Details
* Related to https://github.com/ClickHouse/clickhouse-go/issues/955
* Should a `case` for `decimal.Decimal` have been implemented in https://github.com/ClickHouse/clickhouse-go/blob/main/lib/column/nullable.go#L78-L103 ?
Note: seems to be working as expected if I implement a similar test for `TestStdDynamicScan`, in `examples/std/dynamic_scan_types.go`
```
=== RUN TestStdDynamicScan
NULL
NULL
NULL
123.1
NULL
124.1
NULL
NULL
NULL
NULL
--- PASS: TestStdDynamicScan (0.02s)
```
### Environment
* [x] `clickhouse-go` version: 2.40.3
* [x] Interface: ClickHouse API
* [x] Go version: 1.25.4
* [x] Operating system: macOS
* [x] ClickHouse version: 25.11 (latest)
* [x] Is it a ClickHouse Cloud? No
* [x] ClickHouse Server non-default settings, if any:
* [x] `CREATE TABLE` statements for tables involved:
* [x] Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
Contributor guide
Research direction
Reproduce the failure with examples/clickhouse_api/dynamic_scan_types.go and the shown DynamicScan test command, then inspect the nullable scanning logic in lib/column/nullable.go. Compare it with examples/std/dynamic_scan_types.go and TestStdDynamicScan. Done means nullable Decimal64 scans produce NULL for every NULL row without retaining the previous value.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 50/100