ClickHouse / ClickHouse/clickhouse-go

Batch insert does not support `Object(Nullable('json'))`

Open
#1,017 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
3.3k
Forks
680
Avg merge
2d 3h
Merged PRs (30d)
14

Description

### Describe the bug
Batch insert on a table with a column of type `Object(Nullable('json'))` fails with the error:

```
PrepareContext error: clickhouse: unsupported column type "Object(Nullable('json'))"
```

However a table with a column of type `JSON` works.

### Steps to reproduce

1. `clickhouse client -q "create database test"`
2. `go run clickhouse_json_batch.go`

### Expected behaviour

Should be able to batch insert on a table with column of type `Object(Nullable('json'))` similar to `Object('json')`.

### Code example
```go
// clickhouse_json_batch.go

package main

import (
"context"
"encoding/json"

"github.com/ClickHouse/clickhouse-go/v2"
"github.com/ClickHouse/clickhouse-go/v2/lib/driver"
)

func main() {
opts := clickhouse.Options{
Addr: []string{"127.0.0.1:9000"},
Auth: clickhouse.Auth{
Username: "default",
Password: "",
Database: "test",
},
Debug: true,

Settings: clickhouse.Settings{
"allow_experimental_object_type": "1",
},
}

ctx := context.Background()

conn, err := clickhouse.Open(&opts)
if err != nil {
panic(err)
}

if err := conn.Exec(ctx, "CREATE TABLE IF NOT EXISTS onlyjson (col1 JSON) Engine Memory"); err != nil {
panic(err)
}

if err := conn.Exec(ctx, "CREATE TABLE IF NOT EXISTS jsonnull (col1 Object(Nullable('json'))) Engine Memory"); err != nil {
panic(err)
}

batch, err := conn.PrepareBatch(ctx, "INSERT INTO onlyjson")
if err != nil {
panic(err)
}

send(batch)

batch, err = conn.PrepareBatch(ctx, "INSERT INTO jsonnull")
if err != nil {
panic(err)
}

send(batch)
}

func send(batch driver.Batch) {
for i := 0; i < 10; i++ {
data := map[string]any{
"value": i,
}

b, err := json.Marshal(data)
if err != nil {
panic(err)
}

if err := batch.Append(string(b)); err != nil {
panic(err)
}
}

if err := batch.Send(); err != nil {
panic(err)
}
}
```
### Error log

```
[clickhouse][conn=1][127.0.0.1:9000][handshake] -> 0.0.0
[clickhouse][conn=1][127.0.0.1:9000][handshake] <- ClickHouse (ffb0bdf46af5) server version 23.4.2 revision 54462 (timezone UTC)
[clickhouse][conn=1][127.0.0.1:9000][send query] compression="none" CREATE TABLE IF NOT EXISTS onlyjson (col1 JSON) Engine Memory
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][end of stream]
[clickhouse][conn=1][127.0.0.1:9000][send query] compression="none" CREATE TABLE IF NOT EXISTS jsonnull (col1 Object(Nullable('json'))) Engine Memory
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][end of stream]
[clickhouse][conn=1][127.0.0.1:9000][send query] compression="none" INSERT INTO onlyjson VALUES
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][table columns]
[clickhouse][conn=1][127.0.0.1:9000][read data] compression="none". block: columns=1, rows=0
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][read data] compression="none". block: columns=6, rows=12
[clickhouse][conn=1][127.0.0.1:9000][profile events] rows=12
[clickhouse][conn=1][127.0.0.1:9000][read data] compression="none". block: columns=6, rows=16
[clickhouse][conn=1][127.0.0.1:9000][profile events] rows=16
[clickhouse][conn=1][127.0.0.1:9000][end of stream]
[clickhouse][conn=1][127.0.0.1:9000][send query] compression="none" INSERT INTO jsonnull VALUES
[clickhouse][conn=1][127.0.0.1:9000][send data] compression="none"
[clickhouse][conn=1][127.0.0.1:9000][table columns]
[clickhouse][conn=1][127.0.0.1:9000][read data] decode error: clickhouse: unsupported column type "Object(Nullable('json'))"
panic: clickhouse: unsupported column type "Object(Nullable('json'))"
```

### Configuration
#### Environment
* Client version: v2.10.1
* Language version: 1.20.3
* OS: macOS Ventura 13.4
* Interface: native interface

#### ClickHouse server
* ClickHouse Server version: 23.4.2, 23.5.3
* ClickHouse Server non-default settings, if any: None
* `CREATE TABLE` statements for tables involved: See code example

Contributor guide

Open the contributing guide

Research direction

Start at the PrepareBatch and batch.Send entry points exercised by clickhouse_json_batch.go, then compare the JSON and Object(Nullable('json')) column handling shown in the error log. Reproduce against the stated ClickHouse setup, add coverage for the jsonnull table, and verify that batch.Send succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.