ClickHouse / ClickHouse/clickhouse-go
Expose struct column extraction helper for AppendStruct inserts
- Dominant language
- Go
- Stars
- 3.3k
- Forks
- 680
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 14
Description
## Observed
1. Existing issue #587 documents that `AppendStruct` fails when a table contains columns that are absent from the Go struct, even when those columns have server-side `DEFAULT` expressions.
2. The current workaround is to explicitly list the inserted columns in `PrepareBatch`, for example `INSERT INTO events (id, name, timestamp)`, so ClickHouse can apply defaults to omitted columns.
3. That workaround is effective, but it duplicates the struct's `ch` tags in raw SQL. In wide or frequently evolving ingestion structs, the column list can drift from the struct definition.
## Expected behaviour
The driver should expose a small helper that extracts the ordered ClickHouse column names represented by a struct, so callers can build explicit insert column lists from the same source of truth used by `AppendStruct`.
This does not need to change `AppendStruct` semantics or add an implicit `DESCRIBE` round trip. It would make the documented workaround for #587 easier to use correctly.
## Code example
```go
package main
import (
"context"
"fmt"
"strings"
"github.com/ClickHouse/clickhouse-go/v2"
)
type Event struct {
ID uint64 `ch:"id"`
Name string `ch:"name"`
}
func insert(ctx context.Context, conn clickhouse.Conn, events []Event) error {
columns, err := clickhouse.StructColumns(Event{})
if err != nil {
return err
}
batch, err := conn.PrepareBatch(ctx, fmt.Sprintf(
"INSERT INTO events (%s)",
strings.Join(columns, ", "),
))
if err != nil {
return err
}
defer batch.Close()
for i := range events {
if err := batch.AppendStruct(&events[i]); err != nil {
return err
}
}
return batch.Send()
}
```
With a table like this, ClickHouse can apply `source` server-side because the generated insert statement only names `id` and `name`:
```sql
CREATE TABLE events (
id UInt64,
name String,
source String DEFAULT 'server default'
) ENGINE = MergeTree()
ORDER BY id;
```
## Error log
Current open-ended insert behaviour still fails as described in #587:
```log
clickhouse [AppendStruct]: missing destination name "source" in *main.Event
```
## Details
### Environment
* [x] `clickhouse-go` version: current `main` and released v2 versions with `AppendStruct`
* [x] Interface: ClickHouse API
* [x] Go version: not version-specific
* [x] Operating system: not OS-specific
* [x] ClickHouse version: any version where the table has `DEFAULT` columns
* [ ] Is it a ClickHouse Cloud?
* [ ] ClickHouse Server non-default settings, if any:
* [x] `CREATE TABLE` statements for tables involved: see code example above
* [ ] Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
Related: #587
Contributor guide
Research direction
Start with the existing AppendStruct entry point and the struct tag extraction it already uses, then review related issue #587 for the documented failure mode. Add a public helper that returns the ordered ClickHouse column names represented by a struct, and verify that callers can use it to build explicit insert columns without changing AppendStruct semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100