confluentinc / confluentinc/ksql

Move away from using Connect `Struct` internally to represent the key.

Open
#6,232 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
315
Forks
1k
Avg merge
1d 7h
Merged PRs (30d)
36

Description

Move to something more akin to `GenericRow`, i.e. just an array of column values.

Aside from moving us one step closer to removing Connect types from our core system, it also has a performance benefit. `Struct` validates the type of ever field being set, which is expensive, especially for complex types. The Serde have already ensured the types match, so why pay the cost?

Contributor guide

Open the contributing guide

Research direction

No files or tests are named. Trace internal uses of Connect Struct for keys and compare them with the GenericRow-style array representation, starting at the key-handling and Serde paths. Done means the internal key representation no longer depends on Connect Struct while preserving existing key values and Serde behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kafka
Domain
backend, stream-processing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.