ClickHouse / ClickHouse/clickhouse-go
Batch insert does not support `Object(Nullable('json'))`
- Dominant language
- Go
- Stars
- 3.3k
- Forks
- 680
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 14
Description
### Describe the bug
Batch insert on a table with a column of type `Object(Nullable('json'))` fails with the error:
```
PrepareContext error: clickhouse: unsupported column type "Object(Nullable('json'))"
```
However a table with a column of type `JSON` works.
### Steps to reproduce
1. `clickhouse client -q "create database test"`
2. `go run clickhouse_json_batch.go`
### Expected behaviour
Should be able to batch insert on a table with column of type `Object(Nullable('json'))` similar to `Object('json')`.
### Code example
```go
// clickhouse_json_batch.go
package main
import (
"context"
"encoding/json"
"github.com/ClickHouse/clickhouse-go/v2"
"github.com/ClickHouse/clickhouse-go/v2/lib/driver"
)
func main() {
opts := clickhouse.Options{
Addr: []string{"127.0.0.1:9000"},
Auth: clickhouse.Auth{
Username: "default",
Password: "",
Database: "test",
},
Debug: true,
Settings: clickhouse.Settings{
"allow_experimental_object_type": "1",
},
}
ctx := context.Background()
conn, err := clickhouse.Open(&opts)
if err != nil {
panic(err)
}
if err := conn.Exec(ctx, "CREATE TABLE IF NOT EXISTS onlyjson (col1 JSON) Engine Memory"); err != nil {
panic(err)
}
if err := conn.Exec(ctx, "CREATE TABLE IF NOT EXISTS jsonnull (col1 Object(Nullable('json'))) Engine Memory"); err != nil {
panic(err)
}
batch, err := conn.PrepareBatch(ctx, "INSERT INTO onlyjson")
if err != nil {
panic(err)
}
send(batch)
batch, err = conn.PrepareBatch(ctx, "INSERT INTO jsonnull")
if err != nil {
panic(err)
}
send(batch)
}
func send(batch driver.Batch) {
for i := 0; i < 10; i++ {
data := map[string]any{
"value": i,
}
b, err := json.Marshal(data)
if err != nil {
panic(err)
}
if err := batch.Append(string(b)); err != nil {
panic(err)
}
}
if err := batch.Send(); err != nil {
panic(err)
}
}
```
### Error log
```
[clickhouse][conn=1][127.0.0.1:9000][handshake] -> 0.0.0
[clickhouse][conn=1][127.0.0.1:9000][handshake] <- ClickHouse (ffb0bdf46af5) server version 23.4.2 revision 54462 (timezone UTC)
[clickhouse][conn=1][127.0.0.1:9000][send query] compression="none" CREATE TABLE IF NOT EXISTS onlyjson (col1 JSON) Engine Memory
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][end of stream]
[clickhouse][conn=1][127.0.0.1:9000][send query] compression="none" CREATE TABLE IF NOT EXISTS jsonnull (col1 Object(Nullable('json'))) Engine Memory
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][end of stream]
[clickhouse][conn=1][127.0.0.1:9000][send query] compression="none" INSERT INTO onlyjson VALUES
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][table columns]
[clickhouse][conn=1][127.0.0.1:9000][read data] compression="none". block: columns=1, rows=0
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][read data] compression="none". block: columns=6, rows=12
[clickhouse][conn=1][127.0.0.1:9000][profile events] rows=12
[clickhouse][conn=1][127.0.0.1:9000][read data] compression="none". block: columns=6, rows=16
[clickhouse][conn=1][127.0.0.1:9000][profile events] rows=16
[clickhouse][conn=1][127.0.0.1:9000][end of stream]
[clickhouse][conn=1][127.0.0.1:9000][send query] compression="none" INSERT INTO jsonnull VALUES
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][table columns]
[clickhouse][conn=1][127.0.0.1:9000][read data] decode error: clickhouse: unsupported column type "Object(Nullable('json'))"
panic: clickhouse: unsupported column type "Object(Nullable('json'))"
```
### Configuration
#### Environment
* Client version: v2.10.1
* Language version: 1.20.3
* OS: macOS Ventura 13.4
* Interface: native interface
#### ClickHouse server
* ClickHouse Server version: 23.4.2, 23.5.3
* ClickHouse Server non-default settings, if any: None
* `CREATE TABLE` statements for tables involved: See code example
Contributor guide
Research direction
Start at the PrepareBatch and batch.Send entry points exercised by clickhouse_json_batch.go, then compare the JSON and Object(Nullable('json')) column handling shown in the error log. Reproduce against the stated ClickHouse setup, add coverage for the jsonnull table, and verify that batch.Send succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100