apache / apache/pulsar

[sink jdbc-clickhouse] avro (and possibly others) sink is broken

Open
#11,457 6 comments 0 reactions 0 assignees View on GitHub
lifecycle/stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**

When publishing & consuming messages avro & json schema encoding works as expected. After creating a simple clickhouse sink (using config from the docs), the messages get written to the table, but every value is `null`, making it useless. Downgrading to 2.7.2 re-running the test it all works as expected. No avro, db schema changes or sink config changes are needed when downgrading.

**To Reproduce**
Steps to reproduce the behavior:
1. Start up a clickhouse and pulsar-standalone
1. Create a clickhouse sink attached to the test topic
2. Publish some messages with avro schema (i used go client for this)
4. See error

**Expected behavior**

I expect the sink to correctly populate my fields.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with ClickHouse, a Pulsar standalone instance, the JDBC sink, and Avro messages as described. Compare the current behavior with version 2.7.2; done means the sink populates the ClickHouse table fields instead of writing null values.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.