ClickHouse / ClickHouse/clickhouse-go
Support Avro as an insertion format
- Dominant language
- Go
- Stars
- 3.3k
- Forks
- 680
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 14
Description
Hi,
We have an Avro file which is directly consumed from Kafka. We don't want to deserialize Avro to Google struct, and then give it to Clickhouse client to serialize it in Clickhouse proto format.. Because it's extra work on the API.
What we imagine is, supporting Avro as an insertion format, so just passing Avro file content as []byte, and then Clickhouse client will send it. And we will define our insert query as "INSERT INTO table FORMAT AVRO".
Do you think this is something easy to implement?
PS: We are also working on this driver to provide a PR but at the moment we are learning how it works..
Contributor guide
Research direction
Start by tracing how clickhouse-go handles insertion formats and []byte payloads; the issue identifies INSERT INTO table FORMAT AVRO as the intended entry point. Done means Avro file content can be passed directly without conversion through a Google struct and is accepted by ClickHouse using that format query.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, go
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 39/100