apache / apache/datafusion

Optimize concatenation of complex data type, such as list, struct

Open
#16,838 7 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

I've found that the concatenation of complex data types will be the bottleneck when I worked on an internal performance issue.

Image

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points and provides only a screenshot of a performance bottleneck. Start by locating the concatenation implementation for complex data types and reproducing the reported workload with a benchmark. Done means demonstrating an improvement without changing concatenation behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.